Jekyll2023-11-05T14:16:16+00:00/feed.xmlA coding column.My personal take on coding, software design and programming languages.Quo vadis? LLMs for AI-assisted code copilots2023-11-05T14:09:00+00:002023-11-05T14:09:00+00:00/blog/quo-vadis-llms-ai-code-copilot<p>Over the course of the decades, programming has seen major shifts in how the craft is performed. Where people wrote on punch cards which were then sewn into core memory by hand, there was not much in terms of automated assistance. By now, modern IDEs have long provided code completion and can give smart suggestions as to what a software developer might want to do. CI/CD pipelines and automated tests nowadays quickly verify the correctness of the given code (at least to a certain degree).</p>
<p>We currently see another big shift in how software is written unfolding before our eyes. AI-assisted code co-piloting is the newest variation of automation and code-completion that is aimed at making software development easier. Compared to traditional approaches, which used static analysis and the languages scoping rules together with hand-crafted templates, these tools leverage the power of Large Language Models (LLMs) which have been fine-tuned to provide code completion. A major contender in this space is <a href="https://github.com/features/copilot">GitHub Copilot</a>.</p>
<p>In this article, I wish to review a couple of research papers of the last three years to highlight the chances, but also the risks of those technologies and ask: Quo vadis? Where will the field of software development head in the near future, and what do we need to take care of so that this becomes a success story and not a gigantic money pit?</p>
<p>There are three papers in the last few years that truly stand out, and together form a somewhat concerning view on the wide-spread use of AI to write computer-assisted code. The three papers are</p>
<ul>
<li><a href="https://arxiv.org/abs/1910.14374">Existence of Stack Overflow Vulnerabilities in Well-known Open Source Projects</a> (2019) by Md. Masudur Rahman and B M Mainul Hossain <sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">1</a></sup></li>
<li><a href="https://arxiv.org/abs/2211.03622">Do Users Write More Insecure Code with AI Assistants?</a> (2022) by Neil Perry, Megha Srivastava, Deepak Kumar and Dan Boneh <sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">2</a></sup></li>
<li><a href="https://arxiv.org/abs/2305.17493">The Curse of Recursion: Training on Generated Data Makes Models Forget</a> (2023) by Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot and Ross Anderson <sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">3</a></sup></li>
</ul>
<p>Lets begin with the second paper <sup id="fnref:3:1" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">2</a></sup>, which posits the question “Do Users Write More Insecure Code with AI Assistants?”. Unfortunately, the paper can be summarized with a single word: Yes. But it also highlights some more truly concerning facts about AI and LLMs.</p>
<p>Current research on verification of LLMs has led to the notion of “trustworthy AI” and “explainable AI”. The thought was that with explanations, LLMs can become more than just black-boxes that we somehow have to trust to give us the right answers. Being neural networks (NNs), LLMs are just probabilistic models that produce <em>plausible</em> text. They have no notion of <em>knowledge</em> or <em>correctness</em> and can give hilariously wrong results that <em>sound plausible</em>. Especially with ChatGPT, there is a great danger in being overly confident in the output, because ChatGPT is trained to be <em>conversational</em> in tone, but also <em>confident</em> in what it says. Common wisdom under professionals nowadays is to only use LLMs like ChatGPT when one is able to <em>verify</em> the correctness of the output. That this is needed and warranted is highlighted by the very study quoted above. But lets dig into that a bit more, and look at why even getting <em>good explanations</em> cannot increase the confidence in the AI or make AI systems trustworthy.</p>
<p>What the study shows is that humans tend to be more critical of explanations if they <em>perceive</em> themselves as being knowledgeable of a topic, even to the point of being <em>overly</em> critic and not trusting the explanation. However, the study also shows that the less knowledgeable humans are about a topic, the more likely they are to trust any given explanation – even if that explanation is completely and utterly wrong. The study is fascinating in how they generated and presented these fake explanations and how those were received by non-experts (in this case, they used pictures of bids and their classification as well as three groups of people – professional ornithologists, hobby bird watchers and people without any background in birds). In my opinion, this leads to the very interesting possibility of an <em>explanation attack</em>, where LLMs might be trained to maliciously give wrong explanations and guide users towards actions they would otherwise not have undertaken.</p>
<p>Thus, we see that not even <em>explainable</em> AI might be enough to ensure good quality of the code or make the AI trustworthy. But lets assume for a moment that the AI is <em>trustworthy</em> in the sense that is has not been manipulated, but is actually trained to make a best effort to provide high-quality code along with explanations of this code. How does it get the training data?</p>
<p>Lets face it, most code out there in the real world isn’t of the highest quality, most is just average, and as much code below average as above average. Training a model on the real code that is out there will just give us average results. So lets identify high-quality code along with good explanations of the code and train it on that. Stack Overflow has become the <em>de-facto</em> source of information for professional software developers and hobbyists alike, with about 21 millions questions already asked and most of them answered. As of Sept. 2023, during the week it has about 6k - 7.5k questions asked <em>per day</em> and goes down to about 4k on the weekend, highlighting the fact that it is much used by professionals during their work week <sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">4</a></sup>. Suffice to say, SO is an influential source for professional software developers.</p>
<p>However, it is not free of errors. In fact, if we look at the first paper <sup id="fnref:4:1" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">1</a></sup>, we can see that there are several highly ranked questions that do have security issues. These are widely copied over into open-source libraries, and it stands to reason that this flawed code has also found entry into many closed-source / proprietary computer programs.</p>
<p>This highlights two important issues, especially together with the study quoted above: First, explanations are not enough even when they are provided, since they might come flawed sources. Even when the LLM were to provide n explanation that is sourced from SO, that code and explanation might very well be flawed as well. <strong>Automating this process likely increases the velocity with which those security flaws spread</strong> even more. It will also cement the code that is written today as the de-facto standard way of writing code in the future and significantly slow down the speed at which old, outdated practices are phased out. Although I do not like the term <a href="https://en.wikipedia.org/wiki/Cargo_cult_programming">“cargo-cult programming”</a>, the Wikipedia article for it aptly describes the problem of blindly following patterns and copying them without understanding why and where they might be appropriate, creating code that is deeply flawed in the process. We have already seen the problem prior to the advent of AI-assisted code copiloting, and there is a high likelihood that AI suggestions are all too often applied too eagerly without understanding if they are appropriate in that context.</p>
<p>This leads us to the last paper and the important question of how we get rid of those patterns, how we develop new ways to write code and how we can still drive <strong>innovation</strong> and novel paradigms to write code in when faced with a high degree of automation and large amounts of code that are written with the assistance of AI code copilots?</p>
<p>And it doesn’t look good that that front, either.</p>
<p>The last study I cited <sup id="fnref:2:1" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">3</a></sup> shows that there is what they call a “model collapse” when re-training LLMs on their own outputs, i.e. code generated by themselves as part of an code-copilot. The capabilities of LLMS diminish the more they are trained on their own outputs, making genuine human inputs to retrain them on invaluable for the future. But the problem is that text written by LLMs cannot reliably be distinguished from text written by humans. Unless there is a major theoretical breakthrough, this means that with higher and higher adoption levels of AI-assisted code-copilots, more and more inputs to these systems will likely be AI generated, unless one starts the painstaking process of curating code and text that is verifiably written by humans alone.</p>
<p>The authors write “[…] over time we start losing information about the true distribution, which first starts with tails disappearing, and over the generations learned behaviours start converging to a point estimate with very small variance. Furthermore, we show that this process is inevitable, even for cases with almost ideal conditions for long-term learning i.e. no function estimation error. […] Finally, we discuss the broader implications of model collapse. We note that access to the original data distribution is crucial: in learning where the tails of the underlying distribution matter, one needs access to real human-produced data. In other words, the use of LLMs at scale to publish content on the Internet will pollute the collection of data to train them: data about human interactions with LLMs will be increasingly valuable.”<sup id="fnref:2:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">3</a></sup></p>
<p>The details of the study do not matter much in the context of this blog article, but the above paragraphs should give rising concern as to where we are heading in terms of future LLMs.</p>
<h2 id="conclusion">Conclusion</h2>
<p>So where are we headed in the next 5-20 years? Honestly, I don’t know. At least in Europe, there is an increasing demand for regulation of software development because the economic damages of software failures and security holes skyrocket. Thus far, we as a field haven’t done a good job at pro-actively design those regulations, with much of the industry being very much against it. AI offers great increases in productivity, but also comes with additional demand on verification and quality assurance. Thus far, adoption rates far outpace the speed at which we can learn to responsibly use these technologies and mitigate their short-term and long-term impacts. There is already talk about AI regulation, driven mainly by economists and law-makers, and we computer scientists and also software developers need to make sure out voices are heard in order to ensure that the regulation we will eventually get is reasonable and sound and does increase software quality and security, and doesn’t just cost a lot of money in paperwork and cover-your-ass actions.</p>
<p>We should also be wary were software development as a whole is headed, how we still drive innovation and make sure AI assisted code-copilots are a sustainable, safe and high-quality tool hat is able to stick around for a long time. This will involve better quality assurance of those tools as well as strategies for innovation and re-training of these models to allow programming to still evolve in the future and not approach a fixed point.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:4" role="doc-endnote">
<p><a href="https://arxiv.org/abs/1910.14374">Existence of Stack Overflow Vulnerabilities in Well-known Open Source Projects</a> (2019) by Md. Masudur Rahman and B M Mainul Hossain <a href="#fnref:4" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:4:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p><a href="https://arxiv.org/abs/2211.03622">Do Users Write More Insecure Code with AI Assistants?</a> (2022) by Neil Perry, Megha Srivastava, Deepak Kumar and Dan Boneh <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:3:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p><a href="https://arxiv.org/abs/2305.17493">The Curse of Recursion: Training on Generated Data Makes Models Forget</a> (2023) by Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot and Ross Anderson <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:2:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a> <a href="#fnref:2:2" class="reversefootnote" role="doc-backlink">↩<sup>3</sup></a></p>
</li>
<li id="fn:1" role="doc-endnote">
<p>https://sostats.github.io/last30days/ <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Sebastian TeumertOver the course of the decades, programming has seen major shifts in how the craft is performed. Where people wrote on punch cards which were then sewn into core memory by hand, there was not much in terms of automated assistance. By now, modern IDEs have long provided code completion and can give smart suggestions as to what a software developer might want to do. CI/CD pipelines and automated tests nowadays quickly verify the correctness of the given code (at least to a certain degree).Record#with - a thought experiment2020-06-07T14:22:00+00:002020-06-07T14:22:00+00:00/blog/record-with-a-thought-experiment<p>Recently, there were some interesting discussions about <code class="language-plaintext highlighter-rouge">Record#copy()</code> and <code class="language-plaintext highlighter-rouge">Record#with(...)</code> on the <code class="language-plaintext highlighter-rouge">amber-spec-experts</code>
mailing list (<a href="https://mail.openjdk.java.net/pipermail/amber-spec-experts/2020-May/002217.html">1</a>, <a href="https://mail.openjdk.java.net/pipermail/amber-spec-experts/2020-May/002221.html">2</a>) which has lead me to implement both methods with a very clear & typesafe way, which I’d like to
discuss below. The code demonstrated below is available as <a href="https://gist.github.com/NetzwergX/4ebba8ea36d0663f2a540d0f71f16e49">Gist</a>.</p>
<p>What led me to start this experiment was the following quote:</p>
<blockquote>
<p>the idea is to add a method <code class="language-plaintext highlighter-rouge">Record with(Object... componentValuePairs)</code> in <code class="language-plaintext highlighter-rouge">java.lang.Record</code>, and ask the
compiler to verify that the even arguments (0, 2, 4, etc) are constant strings</p>
<p>Proposed syntax:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>var otherPerson = person.with("name", "John", "age", 17);
</code></pre></div> </div>
</blockquote>
<p>I thought this <em>had</em> to be easier and be possible in a typesafe way. So here is my suggestion:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>var otherPerson = person.with(Person::name, "John").with(Person::age, 17);
</code></pre></div></div>
<p>What do we get? Type saftey. Better, easier to read syntax.</p>
<h1 id="can-we-pull-this-off">Can we pull this off?</h1>
<h2 id="yes">Yes</h2>
<p>It turns out, its actually possible to pull this off just as written there. Yes, <code class="language-plaintext highlighter-rouge">Person::name</code> is an accessor method
and doesn’t allow us to <em>set</em> anything directly - but we already know how the component of the record is called when we
look at that – and we also know the type.</p>
<p>The question is, can we get this information from the lambda?
<a href="https://stackoverflow.com/a/35223119/1360803">Turns out we can</a>.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/**
* c.f. https://stackoverflow.com/a/35223119/1360803
*/</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="nc">SerializedLambda</span> <span class="nf">getSerializedLambda</span><span class="o">(</span><span class="nc">Serializable</span> <span class="n">lambda</span><span class="o">)</span>
<span class="kd">throws</span> <span class="nc">NoSuchMethodException</span><span class="o">,</span> <span class="nc">SecurityException</span><span class="o">,</span>
<span class="nc">IllegalAccessException</span><span class="o">,</span> <span class="nc">IllegalArgumentException</span><span class="o">,</span>
<span class="nc">InvocationTargetException</span> <span class="o">{</span>
<span class="kd">final</span> <span class="nc">Method</span> <span class="n">method</span> <span class="o">=</span> <span class="n">lambda</span><span class="o">.</span><span class="na">getClass</span><span class="o">()</span>
<span class="o">.</span><span class="na">getDeclaredMethod</span><span class="o">(</span><span class="s">"writeReplace"</span><span class="o">);</span>
<span class="n">method</span><span class="o">.</span><span class="na">setAccessible</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
<span class="k">return</span> <span class="o">(</span><span class="nc">SerializedLambda</span><span class="o">)</span> <span class="n">method</span><span class="o">.</span><span class="na">invoke</span><span class="o">(</span><span class="n">lambda</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>We can use this to extract everything we need:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="o"><</span><span class="no">T</span><span class="o">,</span> <span class="no">F</span> <span class="kd">extends</span> <span class="nc">Serializable</span> <span class="o">&</span> <span class="nc">Function</span><span class="o"><</span><span class="no">R</span><span class="o">,</span> <span class="no">T</span><span class="o">>></span> <span class="kt">void</span> <span class="nf">with</span><span class="o">(</span><span class="no">F</span> <span class="n">param</span><span class="o">)</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">lambda</span> <span class="o">=</span> <span class="n">getSerializedLambda</span><span class="o">(</span><span class="n">param</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">name</span> <span class="o">=</span> <span class="n">lambda</span><span class="o">.</span><span class="na">getImplMethodName</span><span class="o">();</span>
<span class="kt">var</span> <span class="n">signature</span> <span class="o">=</span> <span class="n">lambda</span><span class="o">.</span><span class="na">getImplMethodSignature</span><span class="o">();</span>
<span class="c1">// get descriptor, strip () of input</span>
<span class="kt">var</span> <span class="n">typeDescriptor</span> <span class="o">=</span> <span class="n">signature</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="n">signature</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">name</span> <span class="o">+</span> <span class="s">":"</span> <span class="o">+</span> <span class="n">typeDescriptor</span><span class="o">);</span>
<span class="c1">// if we call this with with(Person::name), we get `name:Ljava/lang/String;`</span>
<span class="o">}</span>
</code></pre></div></div>
<p>We get the type signature and the name of the accessor method, which is the same as the field. We strip the <code class="language-plaintext highlighter-rouge">()</code> from
the method signature to get the return type.</p>
<p>Putting it all together, we get this nice method:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@SuppressWarnings</span><span class="o">(</span><span class="s">"unchecked"</span><span class="o">)</span>
<span class="kd">public</span> <span class="k">default</span> <span class="o"><</span><span class="no">T</span><span class="o">,</span> <span class="no">F</span> <span class="kd">extends</span> <span class="nc">Serializable</span> <span class="o">&</span> <span class="nc">Function</span><span class="o"><</span><span class="no">R</span><span class="o">,</span> <span class="no">T</span><span class="o">>></span> <span class="no">R</span> <span class="nf">with</span><span class="o">(</span><span class="no">F</span> <span class="n">param</span><span class="o">,</span> <span class="no">T</span> <span class="n">val</span><span class="o">)</span> <span class="o">{</span>
<span class="k">try</span> <span class="o">{</span>
<span class="c1">// get name & type of the changing parameter</span>
<span class="kt">var</span> <span class="n">lambda</span> <span class="o">=</span> <span class="n">getSerializedLambda</span><span class="o">(</span><span class="n">param</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">name</span> <span class="o">=</span> <span class="n">lambda</span><span class="o">.</span><span class="na">getImplMethodName</span><span class="o">();</span>
<span class="kt">var</span> <span class="n">signature</span> <span class="o">=</span> <span class="n">lambda</span><span class="o">.</span><span class="na">getImplMethodSignature</span><span class="o">();</span>
<span class="c1">// get descriptor, strip () of input</span>
<span class="kt">var</span> <span class="n">typeDescriptor</span> <span class="o">=</span> <span class="n">signature</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="n">signature</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
<span class="c1">// get record components & replace the value</span>
<span class="kt">var</span> <span class="n">components</span> <span class="o">=</span> <span class="n">getClass</span><span class="o">().</span><span class="na">getRecordComponents</span><span class="o">();</span>
<span class="kt">var</span> <span class="n">params</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Object</span><span class="o">[</span><span class="n">components</span><span class="o">.</span><span class="na">length</span><span class="o">];</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">components</span><span class="o">.</span><span class="na">length</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">component</span> <span class="o">=</span> <span class="n">components</span><span class="o">[</span><span class="n">i</span><span class="o">];</span>
<span class="k">if</span> <span class="o">(</span><span class="n">isCompatible</span><span class="o">(</span><span class="n">component</span><span class="o">,</span> <span class="n">name</span><span class="o">,</span> <span class="n">typeDescriptor</span><span class="o">))</span>
<span class="n">params</span><span class="o">[</span><span class="n">i</span><span class="o">]</span> <span class="o">=</span> <span class="n">val</span><span class="o">;</span>
<span class="k">else</span> <span class="o">{</span>
<span class="n">params</span><span class="o">[</span><span class="n">i</span><span class="o">]</span> <span class="o">=</span> <span class="n">component</span><span class="o">.</span><span class="na">getAccessor</span><span class="o">().</span><span class="na">invoke</span><span class="o">(</span><span class="k">this</span><span class="o">);</span>
<span class="c1">// accessor might modify data, so circumvent accessor</span>
<span class="c1">// but records don't expose their fields :(</span>
<span class="c1">//params[i] = getClass().getField(component.getName()).get(this);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// create new record</span>
<span class="k">return</span> <span class="o">(</span><span class="no">R</span><span class="o">)</span> <span class="n">getClass</span><span class="o">().</span><span class="na">getConstructors</span><span class="o">()[</span><span class="mi">0</span><span class="o">].</span><span class="na">newInstance</span><span class="o">(</span><span class="n">params</span><span class="o">);</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">NoSuchMethodException</span> <span class="o">|</span> <span class="nc">SecurityException</span>
<span class="o">|</span> <span class="nc">IllegalAccessException</span> <span class="o">|</span> <span class="nc">IllegalArgumentException</span>
<span class="o">|</span> <span class="nc">InvocationTargetException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">RuntimeException</span><span class="o">(</span><span class="n">e</span><span class="o">);</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">InstantiationException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">RuntimeException</span><span class="o">(</span><span class="n">e</span><span class="o">);</span>
<span class="o">}</span> <span class="cm">/*catch (NoSuchFieldException e) {
throw new RuntimeException(e);
}*/</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The full code is available as <a href="https://gist.github.com/NetzwergX/4ebba8ea36d0663f2a540d0f71f16e49">Gist</a>. I’ve created an interface with this method as only default method, which
can be implemented by any record.</p>
<p>And it actually works as advertised:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">Person</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">)</span> <span class="kd">implements</span> <span class="nc">CopyableRecord</span><span class="o"><</span><span class="nc">Person</span><span class="o">></span> <span class="o">{}</span>
<span class="kt">var</span> <span class="n">guruJava</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Brian Goetz"</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">guruCSharp</span> <span class="o">=</span> <span class="n">guruJava</span><span class="o">.</span><span class="na">with</span><span class="o">(</span><span class="nl">Person:</span><span class="o">:</span><span class="n">name</span><span class="o">,</span> <span class="s">"Eric Lippert"</span><span class="o">);</span>
</code></pre></div></div>
<h2 id="but-with-a-few-drawbacks">But with a few drawbacks.</h2>
<p>The first and most obvious drawback: This only works with method references like <code class="language-plaintext highlighter-rouge">Person::firstName</code>. Which is fair, I
guess. This doesn’t work:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">original</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Person</span><span class="o">(</span><span class="s">"Brian"</span><span class="o">,</span> <span class="s">"Goetz"</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">copy</span> <span class="o">=</span> <span class="n">original</span><span class="o">.</span><span class="na">with</span><span class="o">(</span><span class="n">p</span> <span class="o">-></span> <span class="n">p</span><span class="o">.</span><span class="na">firstName</span><span class="o">(),</span> <span class="s">"Eric"</span><span class="o">);</span> <span class="c1">// nope</span>
</code></pre></div></div>
<p>In fact, my current implementation silently ignores this (which is ok for a thought experiment, not so much for
production code).</p>
<p>Furthermore, the method I have presented above has a few commented-out lines.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">params</span><span class="o">[</span><span class="n">i</span><span class="o">]</span> <span class="o">=</span> <span class="n">component</span><span class="o">.</span><span class="na">getAccessor</span><span class="o">().</span><span class="na">invoke</span><span class="o">(</span><span class="k">this</span><span class="o">);</span>
<span class="c1">// accessor might modify data, so circumvent accessor</span>
<span class="c1">// but records don't expose their fields :(</span>
<span class="c1">//params[i] = getClass().getField(component.getName()).get(this);</span>
</code></pre></div></div>
<p>These also lead to the commented-out <code class="language-plaintext highlighter-rouge">NoSuchFieldException</code> catch-block.</p>
<p>So whats the problem? Accessors can change the data. I’m not sure it would be a good or often used design feature a
programmer would <em>want</em> to use to return data that is invalid, instead of just some wrappers around data like
<code class="language-plaintext highlighter-rouge">Collections#unmodifieableList</code>, but it can happen.</p>
<p>Lets look at an – arguably contrived – example:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">Doubling</span> <span class="o">(</span><span class="kt">int</span> <span class="n">n</span><span class="o">,</span> <span class="kt">int</span> <span class="n">m</span><span class="o">)</span> <span class="kd">implements</span> <span class="nc">RecordTransform</span><span class="o"><</span><span class="nc">Doubling</span><span class="o">></span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">int</span> <span class="nf">n</span><span class="o">()</span> <span class="o">{</span> <span class="k">return</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">n</span><span class="o">;</span> <span class="o">}</span>
<span class="kd">public</span> <span class="kt">int</span> <span class="nf">m</span><span class="o">()</span> <span class="o">{</span> <span class="k">return</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">m</span><span class="o">;</span> <span class="o">}</span>
<span class="o">}</span>
<span class="kt">var</span> <span class="n">original</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Doubling</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span> <span class="mi">3</span><span class="o">);</span> <span class="c1">// Doubling[n=2, m=3]</span>
<span class="kt">var</span> <span class="n">copy</span> <span class="o">=</span> <span class="n">original</span><span class="o">.</span><span class="na">with</span><span class="o">(</span><span class="nl">Doubling:</span><span class="o">:</span><span class="n">n</span><span class="o">,</span> <span class="mi">5</span><span class="o">);</span> <span class="c1">// Doubling[n=5, m=6]</span>
</code></pre></div></div>
<p>So yeah. By using the accessor we double the data every time we copy the record.</p>
<h2 id="is-this-really-a-problem">Is this really a problem?</h2>
<p>I’m not so sure. Lets say that we use libraries like ASM to access the underlying fields directly. We can still
trivially easy break copying:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">Doubling</span> <span class="o">(</span><span class="kt">int</span> <span class="n">n</span><span class="o">,</span> <span class="kt">int</span> <span class="n">m</span><span class="o">)</span> <span class="kd">implements</span> <span class="nc">RecordTransform</span><span class="o"><</span><span class="nc">Doubling</span><span class="o">></span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">Doubling</span> <span class="o">{</span>
<span class="n">n</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">n</span><span class="o">;</span>
<span class="n">m</span> <span class="o">=</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">m</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>If we copy the value of the fields, we still double them every time. And circumventing the constructor seems to be a
<em>bad</em> idea. If someone wants to alter drastically different data in the accessor or mutate it this way in their
constructors, why net let them? Sure, copying then has side-effects that might be surprising on the first glance, but
are actually quite logical on second glance.</p>
<p>Whats actually more upsetting in my opinion is that <code class="language-plaintext highlighter-rouge">Record#toString</code> doesn’t use the accessor methods and thus
displays values you can never extract from the record. It will be interesting to see how that plays out with
deconstruction patterns.<br />
If you implement a DoublingRecord like above, what do you expect to happen when you the following?</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">original</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Doubling</span><span class="o">(</span><span class="mi">2</span><span class="o">,</span><span class="mi">3</span><span class="o">);</span>
<span class="n">let</span> <span class="nf">Doubling</span><span class="o">(</span><span class="kt">int</span> <span class="n">m</span><span class="o">,</span> <span class="kt">int</span> <span class="n">n</span><span class="o">)</span> <span class="o">=</span> <span class="n">original</span><span class="o">;</span>
<span class="kt">var</span> <span class="n">copy</span> <span class="o">=</span> <span class="nc">Doubling</span><span class="o">(</span><span class="n">m</span><span class="o">,</span> <span class="n">n</span><span class="o">);</span>
</code></pre></div></div>
<p>The problems you are facing with a <code class="language-plaintext highlighter-rouge">Record#with</code> or <code class="language-plaintext highlighter-rouge">Record#copy</code> method are exactly the same that emerge when talking
about deconstruction. Whatever solution is chosen for deconstruction will also be applicable for those both methods.</p>
<p><small>Personally, I’d think this is a non-issue. If someone wants to implement their records that way, let them and
let them deal with the fallout themselves.</small></p>
<p>My point is: If I’m able to pull this off, I’m sure the Java architects will find a way to implement
<code class="language-plaintext highlighter-rouge">Record#with</code> in a much better way than by using strings to look up values and completely circumventing the type system.
It can be done inside the type system, with type safety & compiler support, and probably a lot more elegant than what
I’ve cobbled up here.</p>
<p><strong>Disclaimer</strong></p>
<p>This is a thought experiment. Do not use this in production. I’ve not benchmarked it, it doesn’t use
<code class="language-plaintext highlighter-rouge">MethodHandle</code>/<code class="language-plaintext highlighter-rouge">VarHandle</code> and is probably not suitable en masse.</p>Sebastian TeumertRecently, there were some interesting discussions about Record#copy() and Record#with(...) on the amber-spec-experts mailing list (1, 2) which has lead me to implement both methods with a very clear & typesafe way, which I’d like to discuss below. The code demonstrated below is available as Gist.Use Jekyll on Windows via the Subsystem for Linux (WSL)2020-05-21T19:36:00+00:002020-05-21T19:36:00+00:00/blog/use-jekyll-on-windows-via-the-subsystem-for-linux-wsl<p>The Windows Subsystem for Linux works surprisingly well by now – at least for simple tasks. In this blog post, I describe how to install the WSL as well as Jekyll (with GitHub Pages support) using Ubuntu as the distro of choice. Installing the WSL and a Linux distro is suprisingly easy and has only 4 steps (one is optional).</p>
<p>The official Jekyll Documentation a section on this <a href="https://jekyllrb.com/docs/installation/windows/#installation-via-bash-on-windows-10">“Jekyll on Windows”</a>, but unfortunately I found the information therein to be outdated.</p>
<ol>
<li><strong>Activate Windows Subsystem</strong>
In order to install the subsystem, you’ll need to open a PowerShell window and type in the following:
<code class="language-plaintext highlighter-rouge">> Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux</code></li>
<li><strong>Install a distro</strong>
Visit the <a href="https://aka.ms/wslstore">Microsoft Store page for Linux Distros</a> and choose a distro. For the sake of this blog post, I’ll use Ubuntu.</li>
<li>Run <code class="language-plaintext highlighter-rouge">sudo apt update && sudo apt upgrade</code></li>
<li><span class="badge">Optional</span> Activate <kbd>Ctrl+<b>Shift</b>+V</kbd> & <kbd>Ctrl+<b>Shift</b>+C</kbd> via <code class="language-plaintext highlighter-rouge">RMB (Mouse) > Properties > Options (Tab) > Use Ctrl+Shift+C/V as Copy/Paste</code></li>
<li>Run <code class="language-plaintext highlighter-rouge">$ lsb_release -a</code>, which should print:
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal
</code></pre></div> </div>
</li>
</ol>
<h2 id="jekyll">Jekyll</h2>
<p>If you have read the aforementioned documentation page of Jekyll, you’ll have found this part:</p>
<blockquote>
<p>Now we can install Ruby. To do this we will use a repository from BrightBox, which hosts optimized versions of Ruby for Ubuntu.</p>
</blockquote>
<p>Unfortunately, BrightBox does not support Ubuntu 20.04 LTS (Focal Fossa), which is what Windows installs when you use the MS Store:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>E: The repository 'http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu focal Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
</code></pre></div></div>
<p>Install <code class="language-plaintext highlighter-rouge">ruby-full</code> instead:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo apt-get install ruby-full
</code></pre></div></div>
<p>And all build essentials, so that we can build gems with native extensions:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo apt install build-essential patch ruby-dev zlib1g-dev liblzma-dev libsqlite3-dev nodejs
</code></pre></div></div>
<p>And finally, for the ease of handling gems, <code class="language-plaintext highlighter-rouge">ruby-bundler</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo apt install ruby-bundler
</code></pre></div></div>
<p>And…. that’s it. You have now a fully everything needed to get going.</p>
<h3 id="creating-your-first-site">Creating your first site</h3>
<p>Lets get the first jekyll site going. The easiest way to set up Jekyll is using the GitHub-Pages gem. It is a great choice even if not deploying on GitHub Pages.</p>
<p>First, we create the following <strong>Gemfile</strong>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>source 'https://rubygems.org'
gem 'github-pages', group: :jekyll_plugins
</code></pre></div></div>
<p>And than its just a matter of installing the bundle and running Jekyll:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bundle install
$ bundle exec jekyll serve --watch
</code></pre></div></div>
<p>From time to time, you’ll want to fetch new versions of the <code class="language-plaintext highlighter-rouge">github-pages</code> gem:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bundle update github-pages
</code></pre></div></div>Sebastian TeumertThe Windows Subsystem for Linux works surprisingly well by now – at least for simple tasks. In this blog post, I describe how to install the WSL as well as Jekyll (with GitHub Pages support) using Ubuntu as the distro of choice. Installing the WSL and a Linux distro is suprisingly easy and has only 4 steps (one is optional).A small side note on JEPs 384 and 3592020-05-08T21:10:00+00:002020-05-08T21:10:00+00:00/blog/a-small-side-note-on-jep-384-and-359<p>When reading about records and having read
<a href="https://cr.openjdk.java.net/~briangoetz/amber/pattern-match.html">Pattern Matching for Java, Gavin Bierman and Brian Goetz, September 2018</a>,
it would be easy to assume that pattern matching in <code class="language-plaintext highlighter-rouge">instanceof</code> would also include <em>deconstruction patterns</em>.
This isn’t a preview feature in Java 14 which included records via <a href="https://openjdk.java.net/jeps/359">JEP 359</a>, but was slated for Java 15 with <a href="https://openjdk.java.net/jeps/384">JEP 384</a>.
Unfortunately, this JEP has changed and deconstruction patterns are no longer planned for that JEP.
Records will be re-previewed as-is.</p>Sebastian TeumertWhen reading about records and having read Pattern Matching for Java, Gavin Bierman and Brian Goetz, September 2018, it would be easy to assume that pattern matching in instanceof would also include deconstruction patterns. This isn’t a preview feature in Java 14 which included records via JEP 359, but was slated for Java 15 with JEP 384. Unfortunately, this JEP has changed and deconstruction patterns are no longer planned for that JEP. Records will be re-previewed as-is.Records & their constructors2020-05-08T20:57:00+00:002020-05-08T20:57:00+00:00/blog/records-their-constructors<p>A look at the constructors of records in Java 14, and how one can leverage
the formal parameter list of records to already enable libraries like Jackson to work with records (and in this
example, deserialize JSON to records).</p>
<p>Records will be re-previewed in Java 15 without any changes as part of [JEP 384]
and are expected to become a regular feature of the language in Java 16.</p>
<h2 id="records-can-have-regular-constructors-just-like-classes">Records can have regular constructors, just like classes</h2>
<p>Records can have regular constructor. The constructor without formal parameter list, which is a new
feature in records and helps combat boilerplate by leveraging auto-initialization, but they can also have regular
constructors with a formal parameter list, just like classes.</p>
<p>A simple record might be given as:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span> <span class="o">}</span>
</code></pre></div></div>
<p>And then we can add sanity checks – in this case just using the constructor without formal parameter list.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">FooBar</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">foo</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">foo</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"foo can not be null or blank"</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Without a formal parameter list, values are assigned after the constructor has run.
So one can initialize fields differently and <strong>doesn’t</strong> need to assign them via <code class="language-plaintext highlighter-rouge">this.x = x</code>.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">FooBar</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">foo</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">foo</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"foo can not be null or blank"</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">bars</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span>
<span class="n">bars</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">bars</code> <strong>will</strong> be assigned correctly and a call to <code class="language-plaintext highlighter-rouge">bars()</code> will return the newly created list.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="k">new</span> <span class="nc">FooBar</span><span class="o">(</span><span class="s">"FooBar"</span><span class="o">,</span> <span class="kc">null</span><span class="o">));</span>
<span class="c1">// prints FooBar[foo=FooBar, bars=[]]</span>
</code></pre></div></div>
<p>However, assigning <code class="language-plaintext highlighter-rouge">bars</code> inside the constructor to <code class="language-plaintext highlighter-rouge">this.bars</code> via <code class="language-plaintext highlighter-rouge">this.bars = bars;</code> is <em>heavily discouraged</em>,
in fact so much so that it is a point of active discussion on the amber-spec-experts mailing list to remove access to
fields via <code class="language-plaintext highlighter-rouge">this.x</code> in the canonical constructor altogether (both reads & writes) [<a href="https://mail.openjdk.java.net/pipermail/amber-spec-experts/2020-April/002111.html">amber-spec-experts</a>].</p>
<p>However, you can also use a constructor with a formal parameter list. When doing so, auto-initialization does <em>not</em> work
and values <em>must</em> be assigned in the constructor (since the fields are final):</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nf">FooBar</span> <span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">foo</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">foo</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"foo can not be null or blank"</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">bars</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span>
<span class="n">bars</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span>
<span class="k">this</span><span class="o">.</span><span class="na">foo</span> <span class="o">=</span> <span class="n">foo</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">bars</span> <span class="o">=</span> <span class="n">bars</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Why would you ever want to do that? Because you can put annotations on them. Currently, libraries like Jackson do not
support records, but work to make this happen is underway [3]. Using formal parameter lists for the constructor lets us
put annotations on them and get around that limitation:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nf">FooBar</span><span class="o">(</span>
<span class="nd">@JsonProperty</span> <span class="nc">String</span> <span class="n">foo</span><span class="o">,</span>
<span class="nd">@JsonProperty</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">foo</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">foo</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"foo can not be null or blank"</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">bars</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span>
<span class="n">bars</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span>
<span class="k">this</span><span class="o">.</span><span class="na">foo</span> <span class="o">=</span> <span class="n">foo</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">bars</span> <span class="o">=</span> <span class="n">bars</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Yes, this actually works right now, in Java 14, with Jackson 2.11.0.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="n">record</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nc">String</span> <span class="n">foo</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nf">FooBar</span><span class="o">(</span><span class="nd">@JsonProperty</span><span class="o">(</span><span class="s">"foo"</span><span class="o">)</span> <span class="nc">String</span> <span class="n">foo</span><span class="o">,</span>
<span class="nd">@JsonProperty</span><span class="o">(</span><span class="s">"bars"</span><span class="o">)</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">bars</span><span class="o">)</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">foo</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">foo</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nf">IllegalArgumentException</span><span class="o">(</span><span class="s">"foo can not be null or blank"</span><span class="o">);</span>
<span class="k">if</span> <span class="o">(</span><span class="n">bars</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span>
<span class="n">bars</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span>
<span class="k">this</span><span class="o">.</span><span class="na">foo</span> <span class="o">=</span> <span class="n">foo</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">bars</span> <span class="o">=</span> <span class="n">bars</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">main</span><span class="o">(</span><span class="nc">String</span><span class="o">[]</span> <span class="n">args</span><span class="o">)</span> <span class="kd">throws</span> <span class="nc">JsonProcessingException</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="k">new</span> <span class="nc">ObjectMapper</span><span class="o">()</span>
<span class="o">.</span><span class="na">readValue</span><span class="o">(</span><span class="s">"{\"foo\" : \"foo\", \"bars\": []}"</span><span class="o">,</span> <span class="nc">FooBar</span><span class="o">.</span><span class="na">class</span><span class="o">));</span>
<span class="c1">// prints out FooBar[foo=foo, bars=[]]</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>Sebastian TeumertA look at the constructors of records in Java 14, and how one can leverage the formal parameter list of records to already enable libraries like Jackson to work with records (and in this example, deserialize JSON to records).On records & (im-) mutability2020-05-08T19:24:00+00:002020-05-08T19:24:00+00:00/blog/on-records-im-mutability-and-lesser-known-features<p>Records are sometimes described as immutable, which is unfortunate. Looking at the JEP, we see them being called
“shallowly-immutable”. That is an important distinction.
A record can still be changed in a multitude of ways, and this article sheds some light on the strategies one
can employ to ensure records actually are immutable.</p>
<p>Records will be re-previewed in Java 15 without any changes as part of <a href="https://openjdk.java.net/jeps/384">JEP 384</a>
and are expected to become a regular feature of the language in Java 16.</p>
<p>Lets start with a simple example and how it can be shown to be not immutable:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">NaiveRecord</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Date</span><span class="o">></span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span> <span class="o">}</span>
</code></pre></div></div>
<p>We can show that this record is mutable by simply adding to or removing from the list as returned by <code class="language-plaintext highlighter-rouge">values()</code>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">naive</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">NaiveRecord</span><span class="o">(</span><span class="s">"naive"</span><span class="o">,</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>());</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">naive</span><span class="o">);</span>
<span class="c1">//prints NaiveRecord[name=naive, values=[]]</span>
<span class="n">naive</span><span class="o">.</span><span class="na">values</span><span class="o">().</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">Date</span><span class="o">(</span><span class="mi">99</span><span class="o">,</span> <span class="mo">01</span><span class="o">,</span> <span class="mo">01</span><span class="o">));</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">naive</span><span class="o">);</span>
<span class="c1">// prints NaiveRecord[name=naive, values=[Mon Feb 01 00:00:00 CET 1999]]</span>
</code></pre></div></div>
<p>How can we do it better? We can adress the above problem by making the list immutable,
leveraging the canonical constructor without using a formal paremeter list:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">BetterRecord</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Date</span><span class="o">></span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">BetterRecord</span> <span class="o">{</span>
<span class="n">values</span> <span class="o">=</span> <span class="nc">Collections</span><span class="o">.</span><span class="na">unmodifiableList</span><span class="o">(</span><span class="n">values</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
<p>Now, changes to the record via <code class="language-plaintext highlighter-rouge">record.values().add/remove()</code> aren’t possible anymore,
as they’d throw an <code class="language-plaintext highlighter-rouge">UnsupportedOperationException</code>.
Unfortunately, changes to the record are still possible by writing to the underlying list directly.
If that has escaped or be given away, the record can still experience changes in state (and thus, <code class="language-plaintext highlighter-rouge">hashCode()</code>).</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">original</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><</span><span class="nc">Date</span><span class="o">>();</span>
<span class="n">original</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">Date</span><span class="o">(</span><span class="mi">99</span><span class="o">,</span> <span class="mo">01</span><span class="o">,</span> <span class="mo">01</span><span class="o">));</span>
<span class="kt">var</span> <span class="n">better</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BetterRecord</span><span class="o">(</span><span class="s">"better"</span><span class="o">,</span> <span class="n">original</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"%s hash=%s"</span><span class="o">.</span><span class="na">formatted</span><span class="o">(</span><span class="n">better</span><span class="o">,</span> <span class="n">better</span><span class="o">.</span><span class="na">hashCode</span><span class="o">()));</span>
<span class="c1">// BetterRecord[name=better, values=[Mon Feb 01 00:00:00 CET 1999]] hash=-1516124796</span>
<span class="n">original</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="nc">Date</span><span class="o">(</span><span class="mi">102</span><span class="o">,</span> <span class="mo">01</span><span class="o">,</span> <span class="mo">01</span><span class="o">));</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"%s hash=%s"</span><span class="o">.</span><span class="na">formatted</span><span class="o">(</span><span class="n">better</span><span class="o">,</span> <span class="n">better</span><span class="o">.</span><span class="na">hashCode</span><span class="o">()));</span>
<span class="c1">// BetterRecord[name=better, values=[Mon Feb 01 00:00:00 CET 1999, Fri Feb 01 00:00:00 CET 2002]] hash=1357225607</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"-----"</span><span class="o">);</span>
</code></pre></div></div>
<p>The solution to that particular problem is to defensively copy the whole list, and then wrapping it into an unmodifiable one.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">EvenBetterRecord</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Date</span><span class="o">></span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">EvenBetterRecord</span> <span class="o">{</span>
<span class="n">values</span> <span class="o">=</span> <span class="nc">Collections</span><span class="o">.</span><span class="na">unmodifiableList</span><span class="o">(</span><span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>(</span><span class="n">values</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
<p>But still, this record remains vulnerable if the data type stored in the list is mutable, as is the case for <code class="language-plaintext highlighter-rouge">java.util.Date</code>:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">var</span> <span class="n">original</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><</span><span class="nc">Date</span><span class="o">>();</span>
<span class="kt">var</span> <span class="n">someDay</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Date</span><span class="o">(</span><span class="mi">100</span><span class="o">,</span> <span class="mo">01</span><span class="o">,</span> <span class="mo">01</span><span class="o">);</span>
<span class="n">original</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">someDay</span><span class="o">);</span>
<span class="kt">var</span> <span class="n">evenBetter</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">EvenBetterRecord</span><span class="o">(</span><span class="s">"even better"</span><span class="o">,</span> <span class="n">original</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"%s hash=%s"</span><span class="o">.</span><span class="na">formatted</span><span class="o">(</span><span class="n">evenBetter</span><span class="o">,</span> <span class="n">evenBetter</span><span class="o">.</span><span class="na">hashCode</span><span class="o">()));</span>
<span class="c1">// EvenBetterRecord[name=even better, values=[Tue Feb 01 00:00:00 CET 2000]] hash=-1168472954</span>
<span class="n">someDay</span><span class="o">.</span><span class="na">setYear</span><span class="o">(</span><span class="mi">99</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"%s hash=%s"</span><span class="o">.</span><span class="na">formatted</span><span class="o">(</span><span class="n">evenBetter</span><span class="o">,</span> <span class="n">evenBetter</span><span class="o">.</span><span class="na">hashCode</span><span class="o">()));</span>
<span class="c1">// EvenBetterRecord[name=even better, values=[Mon Feb 01 00:00:00 CET 1999]] hash=1655265406</span>
</code></pre></div></div>
<p>We can address this by also creating a deep copy of each element of the list.
This is where having a copy constructor comes in very handy. Unfortunately, <code class="language-plaintext highlighter-rouge">java.util.Date</code> doesn’t have one.
But it <em>does</em> implement <code class="language-plaintext highlighter-rouge">Cloneable</code> & has a <code class="language-plaintext highlighter-rouge">public clone()</code> method.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="n">record</span> <span class="nf">ImmutableRecord</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Date</span><span class="o">></span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">ImmutableRecord</span> <span class="o">{</span>
<span class="n">values</span> <span class="o">=</span> <span class="n">values</span><span class="o">.</span><span class="na">stream</span><span class="o">()</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="nl">Date:</span><span class="o">:</span><span class="n">clone</span><span class="o">)</span>
<span class="o">.</span><span class="na">map</span><span class="o">(</span><span class="nc">Date</span><span class="o">.</span><span class="na">class</span><span class="o">::</span><span class="n">cast</span><span class="o">)</span>
<span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="nc">Collectors</span><span class="o">.</span><span class="na">toUnmodifiableList</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Here we take the values, clone them, cast them back from <code class="language-plaintext highlighter-rouge">Object</code> to <code class="language-plaintext highlighter-rouge">Date</code> (as <code class="language-plaintext highlighter-rouge">Date#clone</code> returns <code class="language-plaintext highlighter-rouge">Object</code>)
and then collect them all into an unmodifiable list.</p>
<p>This list is not modifiable by any of the tricks shown so far.
We can of course still do shenanigans involving <code class="language-plaintext highlighter-rouge">Unsafe</code> or serialization, but this is the kind of record I’d consider
<em>reasonably immutable</em>. If you are not using <code class="language-plaintext highlighter-rouge">java.util.Date</code>, you might need to make sure that you are actually doing
a deep copy of the object, as a shallow copy might have leaked references through which it can be manipulated, too.</p>
<p>Note that only nesting records is no defense again mutation, either.
If the nested record is mutable, e.g. by having an improperly treated collection, the holding record becomes mutable, too.</p>
<h2 id="recap">Recap</h2>
<p>Records are not immutable, they are only “shallowly-immutable”. We can break immutability in three ways:</p>
<ul>
<li>Adding to or removing from the list returned by <code class="language-plaintext highlighter-rouge">values()</code><br />
(addressed by wrapping it into an unmodifiable list)</li>
<li>Adding or removing from the original list if we still have a reference to it<br />
(addressed by copying the list)</li>
<li>Mutating an element in the list mutates the record<br />
(addressed by deep copying the elements of the list)</li>
</ul>
<p>In fact, we can easily devise unit-tests for all three of these properties.
The following gist contains a JUnit 5 test case, with unit tests called</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">canNotChangeThroughGetter</code>,</li>
<li><code class="language-plaintext highlighter-rouge">canNotChangeThroughOriginalList</code> and</li>
<li><code class="language-plaintext highlighter-rouge">canNotChangeThroughOriginalObject</code>.</li>
</ul>
<p>In order to execute each of these tests with each record, I have added a common interface <code class="language-plaintext highlighter-rouge">RecordWithList</code> to all four
of these records and execute the tests with each type of record.
Not all records pass all tests, which was expected in this case.</p>
<details class="github-gist" data-url="https://gist.github.com/NetzwergX/e0e09f3a10f40bdae7fac643193b8d0e">
<summary>Show Gist</summary>
<script src="https://gist.github.com/e0e09f3a10f40bdae7fac643193b8d0e.js"> </script>
</details>
<h1 id="conclusion">Conclusion</h1>
<p>Using the canonical constructor without formal parameter list can be quite powerful.
Using <code class="language-plaintext highlighter-rouge">this.x</code> inside such a constructor is not needed and actively discouraged to the point
that the Java architects are considering disallowing such access altogether.</p>
<p>Using a formal parameter list on the constructor allows programmers to place annotations there.
With such a constructor, some libraries, most notably Jackson, can be made to work with constructors easily.</p>
<p>Records are only “shallowly-immutable”. If one wants to leverage the beneficial properties of immutable types,
great care has to be taken to ensure that the record actually is immutable, and not only appears that way at first glance.</p>Sebastian TeumertRecords are sometimes described as immutable, which is unfortunate. Looking at the JEP, we see them being called “shallowly-immutable”. That is an important distinction. A record can still be changed in a multitude of ways, and this article sheds some light on the strategies one can employ to ensure records actually are immutable.Failure Modes2020-05-07T00:00:00+00:002020-05-07T00:00:00+00:00/blog/failure-modes<p>Programs often have to deal with less then ideal conditions – intermittent internet connections, hardware dropping in
and out, user input not being reliable, files being corrupted, and many, many more scenarios in which failure is not
only a possibility, but to a certain has to be expected and worked with without fatally crashing the application,
but instead gracefully resuming and informing the user of the problems and allowing them to fix them.<br />
In this article, I’ll talk about the following three ways to handle failure modes.</p>
<h1 id="failure-modes">Failure modes</h1>
<ol>
<li>Return <code class="language-plaintext highlighter-rouge">true</code>, <code class="language-plaintext highlighter-rouge">false</code>, <code class="language-plaintext highlighter-rouge">NULL</code> or another <em>magic</em> value</li>
<li>Use <code class="language-plaintext highlighter-rouge">void</code>, throw a (checked) exception</li>
<li>Return an intermediate result object</li>
</ol>
<p>Each of these modes is discussed below, with examples of their usage, strengths and weaknesses.</p>
<h1 id="return-true-false-or-null">Return <code class="language-plaintext highlighter-rouge">true</code>, <code class="language-plaintext highlighter-rouge">false</code> or <code class="language-plaintext highlighter-rouge">NULL</code></h1>
<p>This is used in some older, mostly C-style APIs. Today, we can for example find it quite often in the
PHP standard library, leading to code like this:</p>
<pre><code class="language-PHP">if ($data = xyz_parse($data) === FALSE)
$error = xyz_last_error();
</code></pre>
<p>Obviously this only works in languages like PHP because there is no static typing and in C due to being able to cast
values this way. In Java, the only way to signal failure would be to return <code class="language-plaintext highlighter-rouge">null</code> or another magic value
(via the NULL-Object pattern).</p>
<p>The downsides to this approach are (not exhaustive):</p>
<ul>
<li>It is not clear from the method signature what the magic values are</li>
<li>Without a-priory knowledge about the magic values, checking for them and interpreting them correctly is impossible</li>
<li><em>No compiler support</em>, forcing the caller to correctly handle all scenarios by themselves, making errors easy</li>
</ul>
<p>In multi-threaded contexts, this approach is even worse. In between parsing and retrieving the error,
another thread might parse something, overwriting the stored errors.
Synchronizing such access is easily forgotten or a performance nightmare.</p>
<p>Therefore, this approach has largely fallen out of favor, especially in programming languages that offer useful alternatives.</p>
<p>We can sometimes see a similar approach being used in Java where no return value is expected and instead a boolean
is returned to indicate failure and success, e.g. in <code class="language-plaintext highlighter-rouge">Collection#add</code>. The disadvatange is that we do not get any
information about the kind of failure and have no way to query what went wrong.</p>
<p>And in case of <code class="language-plaintext highlighter-rouge">Collections.unmodifiable[Collection|List|Set|Map]</code>, we get a runtime exception (<code class="language-plaintext highlighter-rouge">UnsupportedOperationException)</code>,
which by necessity is unchecked. This defeats all the help the compiler could give us and makes unmodifiable collections
prone to let exceptions bubble up the stack without being handled at the proper level.</p>
<h1 id="throw-checked-exceptions">Throw (checked) exceptions</h1>
<p>Checked exceptions are a more elegant way to signal potential failure modes and forcing the caller to handle them.
Since <em>failure is to be expected</em> for certain operations, checked exceptions an be leverage to enforce error handling
by the caller. Java for example uses checked exceptions when dealing with sockets.
The basic assumption is that a call should succeed, but certain operations are known to be unreliable.
Socket connections are among them. So the caller gets forced to handle the failure.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">try</span> <span class="o">{</span>
<span class="n">socket</span><span class="o">.</span><span class="na">open</span><span class="o">();</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">IOException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// deal with the failure, e.g. display error message</span>
<span class="o">}</span>
</code></pre></div></div>
<h1 id="return-result-object">Return result object</h1>
<p>Another way to handle expected failure is to return an immediate result object that holds information about
the operation. In modern Java, <code class="language-plaintext highlighter-rouge">Optional<E></code> could also be used to signal values that might or might not be present
(e.g. a valid parsing result).</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">ValidationResult</span> <span class="n">result</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="na">validate</span><span class="o">(</span><span class="n">rawData</span><span class="o">);</span>
<span class="k">if</span> <span class="o">(</span><span class="n">result</span><span class="o">.</span><span class="na">isValid</span><span class="o">())</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">data</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="na">getData</span><span class="o">();</span>
<span class="c1">// process data</span>
<span class="o">}</span>
<span class="k">else</span> <span class="o">{</span>
<span class="kt">var</span> <span class="n">error</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="na">getValidationError</span><span class="o">();</span>
<span class="c1">// optional error handling</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Nice, clean logic for the caller. In Java, <code class="language-plaintext highlighter-rouge">ValidationResult#getData</code> could return <code class="language-plaintext highlighter-rouge">Optional<BusinessData></code> to
communicate to the caller that the data might or might not be present and to ensure compiler support for missing values.</p>
<h2 id="leveraging-sealed-classes-and-interfaces-algebraic-types">Leveraging sealed classes and interfaces (algebraic types)</h2>
<p>With [JEP 360] <a href="https://mail.openjdk.java.net/pipermail/amber-dev/2020-April/005784.html">considered for Java 15</a>,
we might get yet another way to create intermediate result objects that sits somewhere in between checked exceptions
and a result object - algebraic types! These are formed with <a href="https://cr.openjdk.java.net/~briangoetz/amber/datum.html">Sealed Types</a> and <a href="https://openjdk.java.net/jeps/359">Records</a>.</p>
<p>Suppose we create a type <code class="language-plaintext highlighter-rouge">Result<S, E> = Success<S> | Error<E></code>, which we could do In Java 15 with</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sealed</span> <span class="kd">interface</span> <span class="nc">Result</span><span class="o"><</span><span class="no">S</span><span class="o">,</span><span class="no">E</span><span class="o">></span> <span class="n">permits</span> <span class="nc">Success</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">>,</span> <span class="nc">Error</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">></span> <span class="o">{</span>
<span class="n">record</span> <span class="nc">Success</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">></span> <span class="o">(</span><span class="no">S</span> <span class="n">result</span><span class="o">)</span> <span class="kd">implements</span> <span class="nc">Result</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">></span> <span class="o">{}</span>
<span class="n">record</span> <span class="nc">Error</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">></span> <span class="o">(</span><span class="no">E</span> <span class="n">error</span><span class="o">)</span> <span class="kd">implements</span> <span class="nc">Result</span><span class="o"><</span><span class="no">S</span><span class="o">,</span> <span class="no">E</span><span class="o">></span> <span class="o">{}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Unfortunately, we can not specify a “don’t care” generic wildcard, so a bit repetition is needed there. But now, instead of
indicating to a caller the possibility of errorneos execution via <code class="language-plaintext highlighter-rouge">throws</code>, we can indicate this via the result type:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="nc">Parser</span> <span class="o">{</span>
<span class="kd">public</span> <span class="nc">Result</span><span class="o"><</span><span class="nc">Data</span><span class="o">,</span> <span class="nc">InvalidFormatException</span><span class="o">></span> <span class="nf">parse</span><span class="o">(</span><span class="nc">String</span> <span class="n">raw</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// [...]</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>A caller is then forced to examine the type, either via <code class="language-plaintext highlighter-rouge">instanceof</code> (<a href="https://openjdk.java.net/jeps/305">JEP 305</a>) or leveraging <code class="language-plaintext highlighter-rouge">switch</code> expressions (<a href="https://openjdk.java.net/jeps/325">JEP 325</a>),
potentially even leveraging pattern-matching with deconstruction patterns inside of any of those (<a href="https://openjdk.java.net/jeps/8213076">JEP Draft</a>, <a href="https://cr.openjdk.java.net/~briangoetz/amber/pattern-match.html">Pattern Matching</a>).</p>
<p>An example call could look like this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">switch</span><span class="o">(</span><span class="n">parser</span><span class="o">.</span><span class="na">parse</span><span class="o">(</span><span class="n">input</span><span class="o">))</span> <span class="o">{</span>
<span class="k">case</span> <span class="nf">Result</span><span class="o">(</span><span class="nc">Data</span> <span class="n">data</span><span class="o">)</span> <span class="o">-></span> <span class="n">process</span><span class="o">(</span><span class="n">data</span><span class="o">);</span>
<span class="k">case</span> <span class="nf">Error</span><span class="o">(</span><span class="nc">InvalidFormatException</span> <span class="n">e</span><span class="o">)</span> <span class="o">-></span> <span class="n">displayError</span><span class="o">(</span><span class="n">e</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>We might even use a more general exception and switch more fine-grained on the exception type
(potentially forgoing exhaustiveness checks and necessitating a <code class="language-plaintext highlighter-rouge">default</code> clause).</p>
<h1 id="conclusion">Conclusion</h1>
<p>Failure modes can be handled in different ways depending on wether failure is expected or unexpected. Shown here are
four ways of handling them, and which one to choose depends on the nature of the problem and the context
in which it might occur.</p>
<p>There is a (sometimes) heated debate going on when to use (checked) exceptions and when to use other approaches.
A statement that I found very reasonable wrt. exceptions is the following:</p>
<blockquote>
<p>Checked exceptions are for environment problems. Unchecked exceptions are programming errors.<br />
— Elliotte Rusty Harold, <em>Exceptions: I’m Telling You For The Last Time</em></p>
</blockquote>Sebastian TeumertPrograms often have to deal with less then ideal conditions – intermittent internet connections, hardware dropping in and out, user input not being reliable, files being corrupted, and many, many more scenarios in which failure is not only a possibility, but to a certain has to be expected and worked with without fatally crashing the application, but instead gracefully resuming and informing the user of the problems and allowing them to fix them. In this article, I’ll talk about the following three ways to handle failure modes.Java 8 Lambda Cheat Sheet2020-05-06T00:00:00+00:002020-05-06T00:00:00+00:00/blog/java-8-lambda-cheat-sheet<p class="no-print">Even though Java 8 is just a little over six years old and brought a tremendous revolution to Java,
adoption of lambdas and functional approaches to programming problems is still an ongoing process.<br />
I have introduced numerous people to functional-style programming & lambdas,
and at one point came up with the idea of a small “cheat sheet” only one or two DIN A4 pages in size that you could have
in hand when thinking about functional stuff. This is by no means meant to be a comprehensive introduction,
but a small reference document, and has been received very well, so I thought I’d share it here.</p>
<table>
<thead>
<tr>
<th>Interface<br />& Method</th>
<th>Type</th>
<th>Example</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/util/function/Supplier.html"><code class="language-plaintext highlighter-rouge">Supplier<E></code></a> <br /> <code class="language-plaintext highlighter-rouge">E get()</code></td>
<td><code class="language-plaintext highlighter-rouge">() -> E</code></td>
<td><code class="language-plaintext highlighter-rouge">Supplier<List> factory = ArrayList::new;</code><br /><code class="language-plaintext highlighter-rouge">stream.collect(Collectors.toList(factory));</code></td>
</tr>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/util/function/Consumer.html"><code class="language-plaintext highlighter-rouge">Consumer<E></code></a> <br /><code class="language-plaintext highlighter-rouge">void accept(E e)</code></td>
<td><code class="language-plaintext highlighter-rouge">E -> ()</code></td>
<td><code class="language-plaintext highlighter-rouge">list.foreach(e -> e.frobnicate())</code></td>
</tr>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Runnable.html"><code class="language-plaintext highlighter-rouge">Runnable</code></a> <br /> <code class="language-plaintext highlighter-rouge">void run()</code></td>
<td><code class="language-plaintext highlighter-rouge">() -> ()</code></td>
<td><code class="language-plaintext highlighter-rouge">new Thread(() -> System.sleep(10000)).run();</code></td>
</tr>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/util/function/Function.html"><code class="language-plaintext highlighter-rouge">Function<T,R></code></a><br /><code class="language-plaintext highlighter-rouge">R apply(T t)</code></td>
<td><code class="language-plaintext highlighter-rouge">T -> R</code></td>
<td><code class="language-plaintext highlighter-rouge">stream.map(t -> new R(t));</code></td>
</tr>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/util/function/BiFunction.html"><code class="language-plaintext highlighter-rouge">BiFunction<T,U,R></code></a><br /><code class="language-plaintext highlighter-rouge">R apply(T t, U u)</code></td>
<td><code class="language-plaintext highlighter-rouge">(T, U) -> R</code></td>
<td><code class="language-plaintext highlighter-rouge">stream.reduce(start, accumulator, merge</code>)</td>
</tr>
<tr>
<td><a href="https://docs.oracle.com/javase/8/docs/api/java/util/function/Predicate.html"><code class="language-plaintext highlighter-rouge">Predicate<E></code></a> <br /><code class="language-plaintext highlighter-rouge">boolean test(E e)</code></td>
<td><code class="language-plaintext highlighter-rouge">E -> boolean</code></td>
<td><code class="language-plaintext highlighter-rouge">Function<E, Boolean></code> but with primitive <code class="language-plaintext highlighter-rouge">boolean</code></td>
</tr>
</tbody>
</table>
<h1 id="when-to-use">When to use</h1>
<table>
<thead>
<tr>
<th>Use</th>
<th>When</th>
<th>Related concept</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">Supplier<E></code></td>
<td>If it takes nothing</td>
<td>Factory</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">Consumer<E></code></td>
<td>If it returns nothing</td>
<td>Listener, Callback</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">Runnable</code></td>
<td>If it does neither</td>
<td>Task</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">Function<F, T></code></td>
<td>If it does both</td>
<td>Callback</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">BiFunction<T, U , R></code></td>
<td>If it takes two and returns one</td>
<td> </td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">Predicate<E></code></td>
<td>To check semantic properties</td>
<td>Condition</td>
</tr>
</tbody>
</table>
<h1 id="mapreduceimportant-stream-operations">MapReduce<br /><small>Important stream operations</small></h1>
<p><em>As always, <code class="language-plaintext highlighter-rouge">A = B</code> is permissible!</em></p>
<table>
<thead>
<tr>
<th>Operation</th>
<th>Type</th>
<th>Input</th>
<th>Output</th>
<th>See also</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">map</code></td>
<td><code class="language-plaintext highlighter-rouge">A -> B</code></td>
<td><code class="language-plaintext highlighter-rouge">Stream<A></code></td>
<td><code class="language-plaintext highlighter-rouge">Stream<B></code></td>
<td><code class="language-plaintext highlighter-rouge">flatMap</code></td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">filter</code></td>
<td><code class="language-plaintext highlighter-rouge">A -> bool</code></td>
<td><code class="language-plaintext highlighter-rouge">Stream<A></code></td>
<td><code class="language-plaintext highlighter-rouge">Stream<A></code></td>
<td><code class="language-plaintext highlighter-rouge">takeWhile</code></td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">reduce</code></td>
<td><code class="language-plaintext highlighter-rouge">(B, ((B, A) -> B), ((B, B) -> B)) -> B</code></td>
<td><code class="language-plaintext highlighter-rouge">Stream<A></code></td>
<td><code class="language-plaintext highlighter-rouge">B</code></td>
<td><code class="language-plaintext highlighter-rouge">collect</code> is <code class="language-plaintext highlighter-rouge">reduce</code></td>
</tr>
</tbody>
</table>
<h1 id="when-to-use-1">When to use</h1>
<table>
<thead>
<tr>
<th>Use</th>
<th>When</th>
</tr>
</thead>
<tbody>
<tr>
<td><code class="language-plaintext highlighter-rouge">map</code></td>
<td>transforming one Stream into another Stream</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">filter</code></td>
<td>excluding elements from one stream based on a condition (predicate)</td>
</tr>
<tr>
<td><code class="language-plaintext highlighter-rouge">reduce</code></td>
<td>making the stream “smaller” (reducing it), e.g. <br /> - collecting <em>multiple</em> elements into <em>one</em> list<br />- taking a sum, maximum or minimum<br />- or otherwise reducing multiple elements to one element</td>
</tr>
</tbody>
</table>Sebastian TeumertEven though Java 8 is just a little over six years old and brought a tremendous revolution to Java, adoption of lambdas and functional approaches to programming problems is still an ongoing process. I have introduced numerous people to functional-style programming & lambdas, and at one point came up with the idea of a small “cheat sheet” only one or two DIN A4 pages in size that you could have in hand when thinking about functional stuff. This is by no means meant to be a comprehensive introduction, but a small reference document, and has been received very well, so I thought I’d share it here.How To: Use language variables2013-06-09T00:00:00+00:002013-06-09T00:00:00+00:00/blog/How-To-use-language-variables<p>WCF 2.0 comes with built-in internationalization (i18n) or, as some call
it, multi-language support. I18n within WCF 2.0 is realied using so called
<em>language variables</em>. Those variables can be filled with values in multiple
languages and even allow adminitrators to translate their whole board to
another language themselves. But how do plugin developers use language variables
correctly?<br />
<a href="/2013-06-08-how-to-create-wcf-plugins.html">In the last article</a>,
where I wrote about creating a simple WCF 2.0 pugin, I presented a template in
which the term “Hello, World!” was hard coded into the template in english.
This is not only strongly discouraged, but is also dangerous in terms of
encoding and escaping. Languages variables ansure the proper ecspaing of HTML
entities and much more, therefore one should always use language variables over
hard coded text.</p>
<h4 id="modifiying-the-template">Modifiying the template</h4>
<p>In templates, we can use the <code class="language-plaintext highlighter-rouge">{lang}...{/lang}</code> template tag to render the
contents of a language variable. if the variable is not found, the name of the
variable is put out instead. So, a modified version of the template from
the last article would look like this:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{include file='documentHeader'}
<span class="nt"><head></span>
<span class="nt"><title></span>{lang}wcf.page.helloworld.title{/lang} - {PAGE_TITLE|language}<span class="nt"></title></span>
{include file='headInclude' sandbox=false}
<span class="nt"></head></span>
<span class="nt"><body</span> <span class="na">id=</span><span class="s">"tpl{$templateName|ucfirst}"</span><span class="nt">></span>
{include file='header'}
<span class="nt"><header</span> <span class="na">class=</span><span class="s">"boxHeadline"</span><span class="nt">></span>
<span class="nt"><h1></span>{lang}wcf.page.helloworld.title{/lang}<span class="nt"></h1></span>
<span class="nt"></header></span>
{include file='userNotice'}
{lang}wcf.page.helloworld.info{/lang}
{include file='footer'}
<span class="nt"></body></span>
<span class="nt"></html></span>
</code></pre></div></div>
<p>Here we use two language variables – <code class="language-plaintext highlighter-rouge">wcf.page.helloworld.title</code> for the
page title, which is also used in the heading, and <code class="language-plaintext highlighter-rouge">wcf.page.helloworld.info</code>
which will hold the text in the blue information box. You might notice that I
have removed the whole <code class="language-plaintext highlighter-rouge"><p class='info'>...</p></code> box altogether. This is right –
you can use HTML and even template scripting inside language variables.</p>
<p>By now, the page will look like this:</p>
<p><img src="/assets/images/Hello_World_Page_LVars.png" alt="Screenshots of HelloWorld-Page" title="Hello, World! using language variables" /></p>
<h4 id="deploying-language-variables">Deploying language variables</h4>
<p>In order to define values for our language variables, we will use the <strong>Language-
PIP</strong>, which is XML-based. Therefore, we create a new folder named <code class="language-plaintext highlighter-rouge">lang/</code> in
our plugin, and place two files there, namely <code class="language-plaintext highlighter-rouge">de.xml</code> and <code class="language-plaintext highlighter-rouge">en.xml</code>.</p>
<p>The <code class="language-plaintext highlighter-rouge">lang/en.xml</code> file will look like this:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><language</span> <span class="na">xmlns=</span><span class="s">"http://www.woltlab.com"</span> <span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span> <span class="na">xsi:schemaLocation=</span><span class="s">"http://www.woltlab.com http://www.woltlab.com/XSD/maelstrom/language.xsd"</span> <span class="na">languagecode=</span><span class="s">"en"</span> <span class="na">languagename=</span><span class="s">"English"</span> <span class="na">countrycode=</span><span class="s">"en"</span><span class="nt">></span>
<span class="nt"><category</span> <span class="na">name=</span><span class="s">"wcf.page"</span><span class="nt">></span>
<span class="nt"><item</span> <span class="na">name=</span><span class="s">"wcf.page.helloworld.title"</span><span class="nt">></span><span class="cp"><![CDATA[Hello, World!]]></span><span class="nt"></item></span>
<span class="nt"><item</span> <span class="na">name=</span><span class="s">"wcf.page.helloworld.info"</span><span class="nt">></span><span class="cp"><![CDATA[<p class="info">my first page!</p>]]></span><span class="nt"></item></span>
<span class="nt"></category></span>
<span class="nt"></language></span>
</code></pre></div></div>
<p>Like in the <code class="language-plaintext highlighter-rouge">package.xml</code> file, we again declare the used sheme, which in this
case is the scheme for language files. Moreover, we declare for which language
this file is used - in this case english.</p>
<p>by now you will most probably have notices the <code class="language-plaintext highlighter-rouge"><categories></code> block. Language
variables withing WCF are separated into categories. The name of a language
variable alsways has to be prefixed with it’s category. It is most important
always to use the correct category, as unexpected things can happen if you use
the wrong language category (this was cause for much trouble for early WCf 1.x
plugins). Unfortunately, as of now there is no documentation about the
existing language categories available. Therefore, you will need to take a look
into the WCF database youself.</p>
<p>In this case, <code class="language-plaintext highlighter-rouge">wcf.page</code> is the correct category.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><instruction type="language">language/*.xml</instruction>
</code></pre></div></div>Sebastian TeumertWCF 2.0 comes with built-in internationalization (i18n) or, as some call it, multi-language support. I18n within WCF 2.0 is realied using so called language variables. Those variables can be filled with values in multiple languages and even allow adminitrators to translate their whole board to another language themselves. But how do plugin developers use language variables correctly? In the last article, where I wrote about creating a simple WCF 2.0 pugin, I presented a template in which the term “Hello, World!” was hard coded into the template in english. This is not only strongly discouraged, but is also dangerous in terms of encoding and escaping. Languages variables ansure the proper ecspaing of HTML entities and much more, therefore one should always use language variables over hard coded text.How to: Create WCF 2 Plugins2013-06-08T00:00:00+00:002013-06-08T00:00:00+00:00/blog/how-to-create-wcf2-plugins<p>Customizing your WBB4 / WCF2 installation is usually done via plugins. Editing
files, following bogus installation and hacking instructions, this is all long
gone. Since WBB3, which was built on top of WCF1, hacking is obsolete and was
replaced by simply installing plugins via mouse-click in the ACP. The general
principle is the same for WCF2 as it was for WCF1, but some detailes have
changed.</p>
<h3 id="so-what-is-a-plugin">So what is a <em>plugin</em>?</h3>
<p>Basically, just a TAR- or TGZ- archive that contains
some files in a specific structure. The heart of each plugin is the
<code class="language-plaintext highlighter-rouge">package.xml</code>, a configuration file which defines the plugins dependencies and
it’s delivered functionality. The <code class="language-plaintext highlighter-rouge">package.xml</code> file is placed in the root of
the plugin archive.</p>
<h3 id="a-typical-packagexml">A typical <em>package.xml</em></h3>
<p>Typically, a package.xml would look like this:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><package</span> <span class="na">name=</span><span class="s">"org.example.wcf.plugin"</span> <span class="na">xmlns=</span><span class="s">"http://www.woltlab.com"</span> <span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span> <span class="na">xsi:schemaLocation=</span><span class="s">"http://www.woltlab.com http://www.woltlab.com/XSD/maelstrom/package.xsd"</span><span class="nt">></span>
<span class="nt"><packageinformation></span>
<span class="nt"><packagename></span>WCF2 Example Plugin<span class="nt"></packagename></span>
<span class="nt"><packagedescription></span>This plugin is used for demonstrational purposes.<span class="nt"></packagedescription></span>
<span class="nt"><version></span>1.0.0 Alpha 1<span class="nt"></version></span>
<span class="nt"><date></span>2013-08-09<span class="nt"></date></span>
<span class="nt"></packageinformation></span>
<span class="nt"><authorinformation></span>
<span class="nt"><author></span>Sebastian Teumert<span class="nt"></author></span>
<span class="nt"><authorurl></span>http://www.teumert.net/wcf/<span class="nt"></authorurl></span>
<span class="nt"></authorinformation></span>
<span class="nt"><requiredpackages></span>
<span class="nt"><requiredpackage</span> <span class="na">minversion=</span><span class="s">"2.0.0 Alpha 1"</span><span class="nt">></span>com.woltlab.wcf<span class="nt"></requiredpackage></span>
<span class="nt"></requiredpackages></span>
<span class="nt"><instructions</span> <span class="na">type=</span><span class="s">"install"</span><span class="nt">></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"file"</span><span class="nt">></span>files.tar<span class="nt"></instruction></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"template"</span><span class="nt">></span>templates.tar<span class="nt"></instruction></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"language"</span><span class="nt">></span>language/*.xml<span class="nt"></instruction></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"option"</span><span class="nt">></span>option.xml<span class="nt"></instruction></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"eventListener"</span><span class="nt">></span>eventListener.xml<span class="nt"></instruction></span>
<span class="nt"></instructions></span>
<span class="nt"></package></span>
</code></pre></div></div>
<p>So what is this all about? In the first line, the XML version and encoding is
specified. This is pretty straight-forward. In the next line, the package
identifier is specified, in this case <code class="language-plaintext highlighter-rouge">org.example.wcf.plugin</code>. You can call
your plugin whatever you like, but the accepted standard is to use the scheme
<code class="language-plaintext highlighter-rouge">tld.domain.(wcf|wbb|<sa>).<plugin></code>, where <code class="language-plaintext highlighter-rouge"><sa></code> stands for an arbitrary
standalone application. If you don’t know (yet) what it means, don’t worry, you
can wrap around your head around that later, and <code class="language-plaintext highlighter-rouge"><plugin></code> somehow further
categoryzes your plugin. <em>For example</em>, if you happen to write a <code class="language-plaintext highlighter-rouge">FooBar</code>
bbcode, you could name your plugin <code class="language-plaintext highlighter-rouge">tld.domain.wcf.bbcode.foobar</code>.<br />
The remaining attributes, namely <code class="language-plaintext highlighter-rouge">xmlns</code>, <code class="language-plaintext highlighter-rouge">xmlns:xsi</code> and <code class="language-plaintext highlighter-rouge">xsi:schemaLocation</code>
just specify that this XML file follows the structure that is appropriate
for an <code class="language-plaintext highlighter-rouge">package.xml</code>.</p>
<p>Inside the <code class="language-plaintext highlighter-rouge"><packageinformation></code> block, the package name as it can be read in
the package list in the WCF is specified, along with a brief summary of the
funtions of the plugin (e.g. “Provides a FooBar BBCode with uses the FooService
to bar the baz”), the date of creation and the version number. Similarly, the
<code class="language-plaintext highlighter-rouge"><authorinformation></code> block is quite self-explaining.</p>
<h4 id="requirements">Requirements</h4>
<p>Now, the first block that probably needs a more in-detail explanation is the
<code class="language-plaintext highlighter-rouge"><requiredpackages></code> block. In this block, you specify the <em>dependencies</em> of
your plugin. This means that the package is required for your plugin, meaning
it has to be installed in the version specified as <code class="language-plaintext highlighter-rouge">minversion</code>, or higher.</p>
<p>Obviously, the first package to be declared is almost ever <code class="language-plaintext highlighter-rouge">com.woltlab.wcf</code>
or, if you write a WBB plugin, <code class="language-plaintext highlighter-rouge">com.woltlab.wbb</code>. You should take a good look
at which packages are needed for your plugin to work and require them here.</p>
<p>There is also the possibility to <em>exclude</em> incompatible packages - but more on
that will come in a separate article.</p>
<h4 id="instructions-pips">Instructions (PIPs)</h4>
<p>Your plugin will most likely try to deliver some functionality - this is
where the <code class="language-plaintext highlighter-rouge"><instructions></code> blocks come into play. There are two types of
instructions - <em>install</em> and <em>update</em>, with the obvious semantics. You can
specify various <em>update</em> instructions, each with a different <code class="language-plaintext highlighter-rouge">fromversion</code>
attribute to handle updates from other versions of your plugin to the
current, but you should only declare one (1) <em>install</em> block.</p>
<p>Functionality is delivered via so-called <strong>PackageInstallationPlugins</strong> (PIPs).
There are three types of PIPs: file-based, XML-based and script-based PIPs.</p>
<p><em>File-based</em> PIPs usually log which files they delivered and integrate those
files into the installation, e.g. The File-PIP or the Template-PIP (as well as
the ACP-Template-PIP). The Files- and Template-PIP are therefore the two
most important PIPs inside WCF.</p>
<p><em>XML-based</em> PIPs extract the delivered XML files and configure the installation
accordingly. you can for example deliver new ACP options via the Options-PIP,
new user group permissions via the UserGroupOptions-PIP and more. Sometimes
PIPs have correlations, for example if you deliver an EventListener (EL) via the
EventListener-PIP (an XML-based PIP), you need to include the proper PHP
class file in which the code of your EL resides in the Files-PIP.<br />
The Language-PIP is a kind of special PIP, because it does not specify
a single XML file, but allows you to use wildcards in order to recognise
XML files for various languages at once.</p>
<p>The last class of PIPs are the <em>script-based</em> PIPs. They are rarely ever used,
mostly only during installation or major updates. They allow you to specify
additinal scripts that should run during installation or update, making it
possible to do adjustements in the installation that were otherwise impossible
(e.g. rewriting the database structure and keeping the data intact when the
new version of your plugin has a heavily altered database structure). There is
also an <em>SQL-PIP</em>, that allows you to deliver custom .sql files that have to be
executed upton installation. I considers this to be also an script-based PIP.</p>
<h2 id="a-small-sample-plugin">A small sample plugin</h2>
<p>Now, let’s create a small sample plugin that adds a new static page to your WCF
installation. Later on, we will also add an item to the page menu and read some
data from the database that we will display.</p>
<h4 id="the-packagexml">The package.xml</h4>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
<span class="nt"><package</span> <span class="na">name=</span><span class="s">"org.example.wcf.page.helloworld"</span> <span class="na">xmlns=</span><span class="s">"http://www.woltlab.com"</span> <span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span> <span class="na">xsi:schemaLocation=</span><span class="s">"http://www.woltlab.com http://www.woltlab.com/XSD/maelstrom/package.xsd"</span><span class="nt">></span>
<span class="nt"><packageinformation></span>
<span class="nt"><packagename></span>Hello, World!<span class="nt"></packagename></span>
<span class="nt"><packagedescription></span>A variant of he Hello, World! sample for WCF 2.0<span class="nt"></packagedescription></span>
<span class="nt"><version></span>1.0.0 Alpha 1<span class="nt"></version></span>
<span class="nt"><date></span>2013-06-08<span class="nt"></date></span>
<span class="nt"></packageinformation></span>
<span class="nt"><authorinformation></span>
<span class="nt"><author></span>Sebastian Teumert<span class="nt"></author></span>
<span class="nt"><authorurl></span>http://www.teumert.net/wcf/<span class="nt"></authorurl></span>
<span class="nt"></authorinformation></span>
<span class="nt"><requiredpackages></span>
<span class="nt"><requiredpackage</span> <span class="na">minversion=</span><span class="s">"2.0.0 Alpha 1"</span><span class="nt">></span>com.woltlab.wcf<span class="nt"></requiredpackage></span>
<span class="nt"></requiredpackages></span>
<span class="nt"><instructions</span> <span class="na">type=</span><span class="s">"install"</span><span class="nt">></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"file"</span><span class="nt">></span>files.tar<span class="nt"></instruction></span>
<span class="nt"><instruction</span> <span class="na">type=</span><span class="s">"template"</span><span class="nt">></span>templates.tar<span class="nt"></instruction></span>
<span class="nt"></instructions></span>
<span class="nt"></package></span>
</code></pre></div></div>
<p>Pretty straigtforward, huh?</p>
<h4 id="creating-our-own-page-class">Creating our own page class</h4>
<p>The next thing to do is to create our own page. In WCF, all pages have to implement
<code class="language-plaintext highlighter-rouge">wcf\page\IPage</code>, but for simplicity we can also extend <code class="language-plaintext highlighter-rouge">wcf\page\AbstractPage</code>, which
provides a sane base implementation for <code class="language-plaintext highlighter-rouge">wcf\page\IPage</code> (Note that <em>interfaces</em> start
with a capital “I” by convention). Furthermore, A pages name has to end on <code class="language-plaintext highlighter-rouge">Page</code> and
it has to be located inside the folder <code class="language-plaintext highlighter-rouge">wcf\lib\page\</code>. Thefore, it has to be in the
<code class="language-plaintext highlighter-rouge">wcf\page</code> namespace. This is due to how the auto-loader of WCF works, it maps namespaces
to folder directly. Furthermore, all PHP files inside WCF can only contain one class
and are named <code class="language-plaintext highlighter-rouge">ClassName.class.php</code>. This means that our page will be called
<code class="language-plaintext highlighter-rouge">HelloWorldPage</code> inside the <code class="language-plaintext highlighter-rouge">wcf\page</code> namespace, will extend <code class="language-plaintext highlighter-rouge">AbstractPage</code> and will
be saved under <code class="language-plaintext highlighter-rouge">lib\page\HelloWorldPage.class.php</code>.</p>
<p>And this is how it would look like:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kn">namespace</span> <span class="nn">wcf\page</span><span class="p">;</span>
<span class="kn">use</span> <span class="no">wcf\system\WCF</span><span class="p">;</span>
<span class="cd">/**
* Shows the Hello, world! page.
*
* @author Sebastian Teumert
* @copyright 2013 teumert.net
* @license
* @package org.example.wcf.page.helloworld
*/</span>
<span class="kd">class</span> <span class="nc">HelloWorldPage</span> <span class="kd">extends</span> <span class="nc">AbstractPage</span> <span class="p">{</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And that is it. We do not override any of the methods of <code class="language-plaintext highlighter-rouge">AbstractPage</code> here, since
we do not need it (yet). Later on, we will override <code class="language-plaintext highlighter-rouge">assignVariables()</code> to activate
our own page menu item, override <code class="language-plaintext highlighter-rouge">readData()</code> to read some data from database and
much more.</p>
<p>Note that we also did not specify a template here. This is due to the auto-detection
used withing WCF. You can specify your own template by using the class member
<code class="language-plaintext highlighter-rouge">$template</code>, but we can rely on the auto-detection feature here which will use the
<code class="language-plaintext highlighter-rouge">helloWorld</code> template for our page automagically.</p>
<h4 id="a-simplistic-template">A simplistic template</h4>
<p>For a simple template, the boilerplate is quite extensive, it should – in general –
look like this:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{include file='documentHeader'}
<span class="nt"><head></span>
<span class="nt"><title></span>Hello, world! - {PAGE_TITLE|language}<span class="nt"></title></span>
{include file='headInclude' sandbox=false}
<span class="nt"></head></span>
<span class="nt"><body</span> <span class="na">id=</span><span class="s">"tpl{$templateName|ucfirst}"</span><span class="nt">></span>
{include file='header'}
<span class="nt"><header</span> <span class="na">class=</span><span class="s">"boxHeadline"</span><span class="nt">></span>
<span class="nt"><h1></span>Hello, world!<span class="nt"></h1></span>
<span class="nt"></header></span>
{include file='userNotice'}
<span class="nt"><p</span> <span class="na">class=</span><span class="s">"info"</span><span class="nt">></span>My first WCF page!<span class="nt"></p></span>
{include file='footer'}
<span class="nt"></body></span>
<span class="nt"></html></span>
</code></pre></div></div>
<p>Save this in <code class="language-plaintext highlighter-rouge">templates\helloWorld.tpl</code>.</p>
<p>For testing purposes, we don’t even need to create a plugin. Lets say <code class="language-plaintext highlighter-rouge">WCF_DIR</code>
is the directoy in which you have a working development version of WCF 2.0, then you
can simply copy the <code class="language-plaintext highlighter-rouge">HelloWorld.class.php</code> to <code class="language-plaintext highlighter-rouge">WCF_DIR\lib\page\HelloWorld.class.php</code>
and the template to <code class="language-plaintext highlighter-rouge">WCF_DIR\templates\helloWorld.tpl</code> and your page will be available
under <code class="language-plaintext highlighter-rouge">index.php/HelloWorld/</code> and should look like this:</p>
<p><img src="/assets/images/Hello_World_Page.png" alt="Screenshots of HelloWorld-Page" title="Hello, World!" /></p>
<h4 id="creating-the-plugin">Creating the plugin</h4>
<p>Now, moving files into an exsiting installation is the hacky way, and it will not
work once XML- or script-based PIPs come into play. Fortunately, we already made
a <code class="language-plaintext highlighter-rouge">package.xml</code> in order to be able to deliver our files as a installable plugin.</p>
<p>Fortunately, building a plugin is quite easy. You only need to place the
<code class="language-plaintext highlighter-rouge">HelloWorldPage.class.php</code> inside a TAR archive (by convention, it should be named
<code class="language-plaintext highlighter-rouge">files.tar</code>. Keep in mind that you will need to preserve the directoy structure,
e.g. you will need to have it placed inside <code class="language-plaintext highlighter-rouge">lib\page\</code> inside the archive.</p>
<p>Then simply pack your template into a separate TAR archive (by convetion, this should
be named <code class="language-plaintext highlighter-rouge">templates.tar</code>).</p>
<p>At last, pack the <code class="language-plaintext highlighter-rouge">files.tar</code>, <code class="language-plaintext highlighter-rouge">templates.tar</code> and <code class="language-plaintext highlighter-rouge">package.xml</code> in an archive and
you are done. You now have an installable package. You can download an already packed
reference version of this plugin under
<a href="/assets/downloads/org.example.wcf.page.helloworld.tar">downloads/org.example.wcf.page.helloworld.tar</a>.</p>
<h4 id="organizing-development-smart-and-building-packages-automagically">Organizing development smart and building packages automagically</h4>
<p>At this point I want to point out <a href="2013-04-28-wmake-wcf2-build-script.html">wmake</a>,
a shell script that I’ve written to automagically build WCF packages.</p>
<p>If you structure your plguin according to WCF standards, you can use wmake to
generate the TAR archive with all relevant files auomatically. Read the linked
article about those standards, they make life much, much easier.</p>
<h2 id="next-steps">Next steps</h2>
<p>In the next few days I’ll post articles in which the base page is extended. At
first I’ll discuss using language variables and delivering them instead of
hard-coding text in the template, followed by page menu item and activation, and
an article about reading data from database and displaying it, as well as an in-depth
overview about WCF 2.0 different built-in page types.</p>
<h2 id="further-information">Further Information</h2>
<p>If you have further questions, tweet me, open up a discussion in the WBB4 beta forums
or the WSF (I read both boards and answer questions about plugin development regularly
there), or <a href="https://github.com/NetzwergX/netzwergx.github.com/issues">file an issue on GitHub</a>.</p>Sebastian TeumertCustomizing your WBB4 / WCF2 installation is usually done via plugins. Editing files, following bogus installation and hacking instructions, this is all long gone. Since WBB3, which was built on top of WCF1, hacking is obsolete and was replaced by simply installing plugins via mouse-click in the ACP. The general principle is the same for WCF2 as it was for WCF1, but some detailes have changed. So what is a plugin? Basically, just a TAR- or TGZ- archive that contains some files in a specific structure. The heart of each plugin is the package.xml, a configuration file which defines the plugins dependencies and it’s delivered functionality. The package.xml file is placed in the root of the plugin archive.