Play Code!
2022-04-27T05:53:41+00:00
https://sangupta.com
Copyright © 2014, Sandeep Gupta
Sandeep Gupta
https://sangupta.com
Parsing Typescript in Go using QuickJS
https://sangupta.com/tech/parsing-typescript-in-go-with-quickjs.html
2022-04-26T00:00:00+00:00
2022-04-26T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>This post is in series where I talk on how to parse Typescript code in <a href="https://sangupta.com/tech/parsing-typescript-in-java.html">Java</a>
and <a href="https://sangupta.com/tech/parsing-typescript-in-go-with-v8go.html">Go using v8go</a>. Instead of using <a href="https://github.com/rogchap/v8go">v8go</a>, this time we will use
<a href="github.com/quickjs-go/quickjs-go">QuickJS-Go</a> to parse and obtain the <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">abstract syntax tree</a>.</p>
<p>As before, there are many use-cases on why this may be required:</p>
<ul>
<li>you want to build a new documentation tool for <code class="language-plaintext highlighter-rouge">Javascript</code>/<code class="language-plaintext highlighter-rouge">Typescript</code></li>
<li>you are building a new version of <a href="https://www.hackerrank.com/">Hacker Rank</a></li>
<li>you want to enable scripting in your own code</li>
</ul>
<p>Back to getting the job done, here are the broader steps:</p>
<ul>
<li>Initialize and load QuickJS</li>
<li>Load Typescript JS library</li>
<li>Load the Typescript code that you would like to parse</li>
<li>And, finally call the TS parser code to obtain the AST</li>
</ul>
<p>Let’s dig into the details. The first step is to initialize a new
QuickJS instance. You would notice the use of <code class="language-plaintext highlighter-rouge">runtime.LockOSThread()</code> -
this is to ensure that QuickJS always operates in the exact same thread.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// this is a must-step as per QuickJS documentation</span>
<span class="n">stdruntime</span><span class="o">.</span><span class="n">LockOSThread</span><span class="p">()</span>
<span class="c">// create new runtime</span>
<span class="n">runtime</span> <span class="o">:=</span> <span class="n">quickjs</span><span class="o">.</span><span class="n">NewRuntime</span><span class="p">()</span>
<span class="k">defer</span> <span class="n">runtime</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
<span class="c">// obtain a new context that we will work with</span>
<span class="n">context</span> <span class="o">:=</span> <span class="n">runtime</span><span class="o">.</span><span class="n">NewContext</span><span class="p">()</span>
<span class="k">defer</span> <span class="n">context</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
</code></pre></div></div>
<p>Once V8 is up, we need to load the Typescript JS code. We make use of a locally
saved <code class="language-plaintext highlighter-rouge">typescript.js</code> file for the same. Read it in memory and then use the <code class="language-plaintext highlighter-rouge">EvalFile</code>
method to load the script.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">tsSource</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">ioutil</span><span class="o">.</span><span class="n">ReadFile</span><span class="p">(</span><span class="s">"/path/to/ts/on/disk/typescript.js"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="nb">panic</span><span class="p">(</span><span class="n">err</span><span class="p">)</span> <span class="c">// panic if load failed</span>
<span class="p">}</span>
<span class="c">// load TS source code</span>
<span class="n">result</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">context</span><span class="o">.</span><span class="n">EvalFile</span><span class="p">(</span><span class="kt">string</span><span class="p">(</span><span class="n">typeScript</span><span class="p">),</span> <span class="m">0</span><span class="p">,</span> <span class="s">"typescript.js"</span><span class="p">)</span>
<span class="n">check</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">result</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
</code></pre></div></div>
<p>Notice the use of <code class="language-plaintext highlighter-rouge">check(err)</code> function call above. It is a convenience function
borrowed from the documentation that allows us to visit the stack/cause in case
something fails inside the QuickJS runtime. The function is as under.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">check</span><span class="p">(</span><span class="n">err</span> <span class="kt">error</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="k">var</span> <span class="n">evalErr</span> <span class="o">*</span><span class="n">quickjs</span><span class="o">.</span><span class="n">Error</span>
<span class="k">if</span> <span class="n">errors</span><span class="o">.</span><span class="n">As</span><span class="p">(</span><span class="n">err</span><span class="p">,</span> <span class="o">&</span><span class="n">evalErr</span><span class="p">)</span> <span class="p">{</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">evalErr</span><span class="o">.</span><span class="n">Cause</span><span class="p">)</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">evalErr</span><span class="o">.</span><span class="n">Stack</span><span class="p">)</span>
<span class="p">}</span>
<span class="nb">panic</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Once Typescript is loaded, we need to build the compiler options to let TS know that
we would like to use the latest syntax version for parsing.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// never free this - throws cgo error at app termination</span>
<span class="n">globals</span> <span class="o">:=</span> <span class="n">context</span><span class="o">.</span><span class="n">Globals</span><span class="p">()</span>
<span class="n">ts</span> <span class="o">:=</span> <span class="n">globals</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"ts"</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">ts</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
<span class="n">scriptTarget</span> <span class="o">:=</span> <span class="n">ts</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"ScriptTarget"</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">scriptTarget</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
<span class="n">system</span> <span class="o">:=</span> <span class="n">scriptTarget</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"Latest"</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">system</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
<span class="n">args</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">([]</span><span class="n">quickjs</span><span class="o">.</span><span class="n">Value</span><span class="p">,</span> <span class="m">4</span><span class="p">)</span>
<span class="n">args</span><span class="p">[</span><span class="m">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">String</span><span class="p">(</span><span class="s">"index.ts"</span><span class="p">)</span>
<span class="n">args</span><span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">String</span><span class="p">(</span><span class="kt">string</span><span class="p">(</span><span class="n">sourceCode</span><span class="p">))</span>
<span class="n">args</span><span class="p">[</span><span class="m">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">String</span><span class="p">(</span><span class="s">""</span><span class="p">)</span>
<span class="n">args</span><span class="p">[</span><span class="m">3</span><span class="p">]</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">Bool</span><span class="p">(</span><span class="no">true</span><span class="p">)</span>
</code></pre></div></div>
<p>Next, obtain the function <code class="language-plaintext highlighter-rouge">createSourceFile</code> from the loaded <code class="language-plaintext highlighter-rouge">ts</code> object. This allows
us to invoke the function directly from Go.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">parseCode</span> <span class="o">:=</span> <span class="n">ts</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"createSourceFile"</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">parseCode</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
</code></pre></div></div>
<p>Load the typescript code that you would like to parse, and then simply use
<code class="language-plaintext highlighter-rouge">context.Call</code> to execute the parser.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sourceCode</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">ioutil</span><span class="o">.</span><span class="n">ReadFile</span><span class="p">(</span><span class="s">"/path/on/disk/typescript/code/index.ts"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="nb">panic</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">result</span><span class="p">,</span> <span class="n">err</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">Call</span><span class="p">(</span><span class="n">globals</span><span class="p">,</span> <span class="n">parseCode</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="n">check</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">result</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
</code></pre></div></div>
<p>If there was no error, <code class="language-plaintext highlighter-rouge">result</code> contains the AST as an object. However, you will need
to iterate over it to convert to a pure Go object or a strongly-typed object. It is left as
an exercise for the reader.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">result</span><span class="o">.</span><span class="n">IsObject</span><span class="p">()</span> <span class="p">{</span>
<span class="c">// print the property names available</span>
<span class="n">names</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">result</span><span class="o">.</span><span class="n">PropertyNames</span><span class="p">()</span>
<span class="n">check</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="s">"Object:"</span><span class="p">)</span>
<span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">name</span> <span class="o">:=</span> <span class="k">range</span> <span class="n">names</span> <span class="p">{</span>
<span class="n">val</span> <span class="o">:=</span> <span class="n">result</span><span class="o">.</span><span class="n">GetByAtom</span><span class="p">(</span><span class="n">name</span><span class="o">.</span><span class="n">Atom</span><span class="p">)</span>
<span class="k">defer</span> <span class="n">val</span><span class="o">.</span><span class="n">Free</span><span class="p">()</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"'%s': %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">val</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">String</span><span class="p">())</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Complete code is available in <a href="https://gist.github.com/sangupta/f08af23ce6816f7fad6ac88a267a1a6f">this gist</a></p>
<p>This concludes the series on different ways to parse Typescript code in <a href="https://sangupta.com/tech/parsing-typescript-in-java.html">Java using J2V8</a>,
<a href="https://sangupta.com/tech/parsing-typescript-in-go-with-v8go.html">Go with v8go</a> and [Go with QuickJS][post3].</p>
<p>Happy Hacking.</p>
Parsing Typescript in Go using V8Go
https://sangupta.com/tech/parsing-typescript-in-go-with-v8go.html
2022-04-24T00:00:00+00:00
2022-04-24T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>In my <a href="https://sangupta.com/tech/parsing-typescript-in-java.html">earlier post</a> we saw how we can parse <a href="https://www.typescriptlang.org/">Typescript</a> into an
<a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a> using <a href="https://github.com/eclipsesource/J2V8">Eclipse J2V8</a>. In this post, we
will explore the same in <a href="https://go.dev">Go</a> using <a href="https://github.com/rogchap/v8go">v8go</a>.</p>
<p>Using <code class="language-plaintext highlighter-rouge">Go</code> has advantages if you are building a tool as you can ship a single
native binary without any depdendency. Though <code class="language-plaintext highlighter-rouge">Java</code> has the convenience of <strong>write once, run many</strong>
but it still requires the presence of <code class="language-plaintext highlighter-rouge">JVM</code> on the user’s machine.</p>
<p>Back to the task at hand, the code requires the following steps:</p>
<ul>
<li>Initialize and load V8</li>
<li>Load Typescript JS library</li>
<li>Load the Typescript code that you would like to parse</li>
<li>And, finally call the TS parser code to obtain the AST</li>
</ul>
<p>Let’s get started. The first step is to initialize a new V8 context.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ctx</span> <span class="o">:=</span> <span class="n">v8</span><span class="o">.</span><span class="n">NewContext</span><span class="p">()</span>
</code></pre></div></div>
<p>Once V8 is up, we need to load the Typescript JS code. We make use of a locally
saved <code class="language-plaintext highlighter-rouge">typescript.js</code> file for the same. Read it in memory and then use the <code class="language-plaintext highlighter-rouge">RunScript</code>
method to load the script.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">tsSource</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">ioutil</span><span class="o">.</span><span class="n">ReadFile</span><span class="p">(</span><span class="s">"/path/to/ts/on/disk/typescript.js"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="nb">panic</span><span class="p">(</span><span class="n">err</span><span class="p">)</span> <span class="c">// panic if load failed</span>
<span class="p">}</span>
<span class="c">// load typescript by converting []byte to string</span>
<span class="n">ctx</span><span class="o">.</span><span class="n">RunScript</span><span class="p">(</span><span class="kt">string</span><span class="p">(</span><span class="n">tsSource</span><span class="p">),</span> <span class="s">"typescript.js"</span><span class="p">)</span>
</code></pre></div></div>
<p>As Typescript is now loaded, we need to build the compiler options to let TS know that
we would like to use the latest syntax version for parsing</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// read global object</span>
<span class="n">obj</span> <span class="o">:=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">Global</span><span class="p">()</span>
<span class="n">typescript</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">obj</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"ts"</span><span class="p">)</span>
<span class="n">ts</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">typescript</span><span class="o">.</span><span class="n">AsObject</span><span class="p">()</span>
<span class="c">// fmt.Println(typescript.IsObject())</span>
<span class="n">moduleKindJs</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">ts</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"ScriptTarget"</span><span class="p">)</span>
<span class="n">moduleKind</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">moduleKindJs</span><span class="o">.</span><span class="n">AsObject</span><span class="p">()</span>
<span class="n">systemJs</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">moduleKind</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"Latest"</span><span class="p">)</span>
<span class="n">system</span> <span class="o">:=</span> <span class="n">systemJs</span><span class="o">.</span><span class="n">String</span><span class="p">()</span>
</code></pre></div></div>
<p>Next, obtain the function <code class="language-plaintext highlighter-rouge">createSourceFile</code> from the loaded <code class="language-plaintext highlighter-rouge">ts</code> object. This allows
us to invoke the function directly from Go.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">fnJs</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">ts</span><span class="o">.</span><span class="n">Get</span><span class="p">(</span><span class="s">"createSourceFile"</span><span class="p">)</span>
<span class="n">fn</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">fnJs</span><span class="o">.</span><span class="n">AsFunction</span><span class="p">()</span>
</code></pre></div></div>
<p>Load the typescript code that you would like to parse:</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// read the source code file</span>
<span class="n">jsFile</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">ioutil</span><span class="o">.</span><span class="n">ReadFile</span><span class="p">(</span><span class="s">"/ts/source/code/on/disk/index.tsx"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="nb">panic</span><span class="p">(</span><span class="n">err</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>We are now all set to invoke the parser. Though this requires setting up a new <code class="language-plaintext highlighter-rouge">isolate</code>
and creating a few wrapper objects to be passed into the function obtained above.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">isolate</span> <span class="o">:=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">Isolate</span><span class="p">()</span>
<span class="n">ctx</span><span class="o">.</span><span class="n">RunScript</span><span class="p">(</span><span class="s">"const compilerOptions = { module: "</span><span class="o">+</span><span class="n">system</span><span class="o">+</span><span class="s">"};"</span><span class="p">,</span> <span class="s">"source-tree.js"</span><span class="p">)</span>
<span class="n">sourceFileName</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">v8</span><span class="o">.</span><span class="n">NewValue</span><span class="p">(</span><span class="n">isolate</span><span class="p">,</span> <span class="s">"index.ts"</span><span class="p">)</span>
<span class="n">sourceCode</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">v8</span><span class="o">.</span><span class="n">NewValue</span><span class="p">(</span><span class="n">isolate</span><span class="p">,</span> <span class="kt">string</span><span class="p">(</span><span class="n">jsFile</span><span class="p">))</span>
<span class="n">compilerOptions</span><span class="p">,</span> <span class="n">_</span> <span class="o">:=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">RunScript</span><span class="p">(</span><span class="s">"compilerOptions"</span><span class="p">,</span> <span class="s">"source-tree.js"</span><span class="p">)</span>
<span class="n">booleanTrue</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">v8</span><span class="o">.</span><span class="n">NewValue</span><span class="p">(</span><span class="n">isolate</span><span class="p">,</span> <span class="no">true</span><span class="p">)</span>
<span class="c">// invoke the parser function</span>
<span class="n">fnValue</span><span class="p">,</span> <span class="n">err</span> <span class="o">:=</span> <span class="n">fn</span><span class="o">.</span><span class="n">Call</span><span class="p">(</span><span class="n">ctx</span><span class="o">.</span><span class="n">Global</span><span class="p">(),</span> <span class="n">sourceFileName</span><span class="p">,</span> <span class="n">sourceCode</span><span class="p">,</span> <span class="n">compilerOptions</span><span class="p">,</span> <span class="n">booleanTrue</span><span class="p">)</span>
</code></pre></div></div>
<p>Check if there was an error while parsing, and if yes, you may want to obtain relevant error
message as well as stack trace on the Javascript side.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="no">nil</span> <span class="p">{</span>
<span class="n">e</span> <span class="o">:=</span> <span class="n">err</span><span class="o">.</span><span class="p">(</span><span class="o">*</span><span class="n">v8</span><span class="o">.</span><span class="n">JSError</span><span class="p">)</span> <span class="c">// JavaScript errors will be returned as the JSError struct</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">Message</span><span class="p">)</span> <span class="c">// the message of the exception thrown</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">Location</span><span class="p">)</span> <span class="c">// the filename, line number and the column where the error occured</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">StackTrace</span><span class="p">)</span> <span class="c">// the full stack trace of the error, if available</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"javascript error: %v"</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span> <span class="c">// will format the standard error message</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Printf</span><span class="p">(</span><span class="s">"javascript stack trace: %+v"</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span> <span class="c">// will format the full error stack trace</span>
<span class="k">return</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If there was no error, <code class="language-plaintext highlighter-rouge">fnValue</code> contains the AST as an object. However, you will need
to iterate over it to convert to a pure Go object or a strongly-typed object. It is left as
an exercise for the reader.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">// following shall return true to indicate it is an object</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">fnValue</span><span class="o">.</span><span class="n">IsObject</span><span class="p">())</span> <span class="c">// returns true</span>
</code></pre></div></div>
<p>Complete code is available in <a href="https://gist.github.com/sangupta/070edef1bc1aec675d47eb3b44ad1b24">this gist</a>.</p>
<p>In my next post we will see how we can achieve the same using <a href="https://bellard.org/quickjs/">QuickJS</a> and its wrapper
for Go, <a href="github.com/quickjs-go/quickjs-go">QuickJS-Go</a>.</p>
<p>Happy Hacking.</p>
Parsing Typescript in Java
https://sangupta.com/tech/parsing-typescript-in-java.html
2022-04-20T00:00:00+00:00
2022-04-20T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>This post talks on how you can parse <a href="https://www.typescriptlang.org/">Typescript</a> code in Java using
<a href="https://github.com/eclipsesource/J2V8">Eclipse V8</a> wrapper. Why would one do that? I used it to prepare
Javascript documentation for <a href="https://bedrock.sangupta.com/">Bedrock</a>, my React component library. I
tried using both <a href="https://react-styleguidist.js.org/">React Styleguidist</a> and <a href="https://storybook.js.org/">Storybook</a>
but haven’t liked both (will discuss those reasons in a later post).</p>
<p>Parsing means generating an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a> from the TS source code.
Once you have the AST you can crawl it, to run any transform or aggregation as
you like.</p>
<p>Let’s start by creating a new V8 instance that also understands the NodeJS runtime</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">NodeJS</span> <span class="n">nodeJS</span> <span class="o">=</span> <span class="nc">NodeJS</span><span class="o">.</span><span class="na">createNodeJS</span><span class="o">();</span>
</code></pre></div></div>
<p>Next step is to load the Typescript parser in V8 using <code class="language-plaintext highlighter-rouge">require</code>. I am using
the entire folder from a previously existing <code class="language-plaintext highlighter-rouge">node_modules</code> folder.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">V8Object</span> <span class="n">typescript</span> <span class="o">=</span> <span class="n">nodeJS</span><span class="o">.</span><span class="na">require</span><span class="o">(</span><span class="k">new</span> <span class="nc">File</span><span class="o">(</span><span class="s">"path_to/node_modules/typescript"</span><span class="o">));</span>
</code></pre></div></div>
<p>Now we need to obtain the <code class="language-plaintext highlighter-rouge">Typescript.ScriptTarget.Latest</code> value.
This value allows us to instruct TS compiler to use latest TS syntax guidelines
during the parsing phase.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">V8Object</span> <span class="n">moduleKind</span> <span class="o">=</span> <span class="n">typescript</span><span class="o">.</span><span class="na">getObject</span><span class="o">(</span><span class="s">"ScriptTarget"</span><span class="o">);</span>
<span class="nc">Integer</span> <span class="n">system</span> <span class="o">=</span> <span class="n">moduleKind</span><span class="o">.</span><span class="na">getInteger</span><span class="o">(</span><span class="s">"Latest"</span><span class="o">);</span>
</code></pre></div></div>
<p>Next, we need to setup compiler options. These shall be needed in the next step.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">V8Object</span> <span class="n">compilerOptions</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">V8Object</span><span class="o">(</span><span class="n">nodeJS</span><span class="o">.</span><span class="na">getRuntime</span><span class="o">());</span>
<span class="n">compilerOptions</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="s">"module"</span><span class="o">,</span> <span class="n">system</span><span class="o">);</span>
</code></pre></div></div>
<p>Before we invoke the parser, let’s load the TS source code in memory from the
file on disk.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">String</span> <span class="n">code</span> <span class="o">=</span> <span class="n">org</span><span class="o">.</span><span class="na">apache</span><span class="o">.</span><span class="na">io</span><span class="o">.</span><span class="na">FileUtils</span><span class="o">.</span><span class="na">readFiletoString</span><span class="o">(</span><span class="s">"my-ts-file.ts"</span><span class="o">);</span>
</code></pre></div></div>
<p>Finally time to invoke the TS compiler:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">V8Object</span> <span class="n">result</span> <span class="o">=</span> <span class="o">(</span><span class="nc">V8Object</span><span class="o">)</span> <span class="n">typescript</span><span class="o">.</span><span class="na">executeJSFunction</span><span class="o">(</span><span class="s">"createSourceFile"</span><span class="o">,</span> <span class="n">fileName</span><span class="o">,</span> <span class="n">code</span><span class="o">,</span> <span class="n">compilerOptions</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
</code></pre></div></div>
<p>Iterating over the <code class="language-plaintext highlighter-rouge">V8Object</code> is difficult, so we will use a utility method
to convert it to normal <code class="language-plaintext highlighter-rouge">java.util.Map</code></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Map</span><span class="o"><</span><span class="nc">String</span><span class="o">,</span> <span class="o">?</span> <span class="kd">super</span> <span class="nc">Object</span><span class="o">></span> <span class="n">astAsMap</span> <span class="o">=</span> <span class="nc">V8ObjectUtils</span><span class="o">.</span><span class="na">toMap</span><span class="o">(</span><span class="n">result</span><span class="o">);</span>
</code></pre></div></div>
<p>And that is all what we need. You may now use <code class="language-plaintext highlighter-rouge">astAsMap</code> to crawl/iterate over the
AST as your use-case desire. But before, we wrap it up here, we need to clean up the
resources that V8 allocated:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="o">(</span><span class="n">nodeJS</span><span class="o">.</span><span class="na">isRunning</span><span class="o">())</span> <span class="o">{</span>
<span class="n">nodeJS</span><span class="o">.</span><span class="na">handleMessage</span><span class="o">();</span>
<span class="o">}</span>
<span class="c1">// release all resources</span>
<span class="n">result</span><span class="o">.</span><span class="na">release</span><span class="o">();</span>
<span class="n">compilerOptions</span><span class="o">.</span><span class="na">release</span><span class="o">();</span>
<span class="n">moduleKind</span><span class="o">.</span><span class="na">release</span><span class="o">();</span>
<span class="n">typescript</span><span class="o">.</span><span class="na">release</span><span class="o">();</span>
<span class="n">nodeJS</span><span class="o">.</span><span class="na">release</span><span class="o">();</span>
</code></pre></div></div>
<p>You can find all the code together in <a href="repo">this Github repository</a> including conversion
to strongly typed Java objects.</p>
Experiments with Hugo!
https://sangupta.com/tech/experiments-with-hugo.html
2022-04-18T00:00:00+00:00
2022-04-18T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>In continuation of my last post, <a href="https://sangupta.com/tech/revisiting-the-blog.html">revisiting this blog</a>, I spent time over weekend
experiment and convert my site to <a href="https://gohugo.io">Hugo</a>. And I was amazed at the very first
run. <code class="language-plaintext highlighter-rouge">Hugo</code> is not just fast, it is super fast. While <a href="https://jekyllrb.com">Jekyll</a> takes
up 3-4 seconds to start, Hugo was done in less than a second. To top it all, it
automatically detects changes to the configuration file as well and rebuilds. And
to add cherries on top of the cake, <code class="language-plaintext highlighter-rouge">Hugo</code> was emitting content in both HTML and
JSON formats to be consumed.</p>
<p>I also love the way Hugo allows you to configure your site either using <code class="language-plaintext highlighter-rouge">YAML</code>, or
<code class="language-plaintext highlighter-rouge">TOML</code> or <code class="language-plaintext highlighter-rouge">JSON</code>. This allows a lot of flexibility for people who may be more
familiar and comfortable using one format than other. I loved the way that at the
flip of a configuration flag, all content (HTML/JS/CSS) was mininfied instantly.
I use <a href="https://mermaid-js.github.io/mermaid/#/">MermaidJS</a> a lot for my sequence diagrams, and loved that <code class="language-plaintext highlighter-rouge">Hugo</code> has
<a href="https://gohugo.io/content-management/diagrams/#mermaid-diagrams">support for the same</a> too.</p>
<p><code class="language-plaintext highlighter-rouge">Hugo</code> has some level of customization in the form of <a href="https://gohugo.io/templates/shortcode-templates/">shortcodes</a>. It has a
decent amount of <a href="https://gohugo.io/functions/">built-in functions</a> that you may leverage. However, I was
disappointed to see no support for date format customization. I was born to read
dates in <code class="language-plaintext highlighter-rouge">dd/MM/yyyy</code> format and now reading <code class="language-plaintext highlighter-rouge">mm/dd/yyyy</code> or <code class="language-plaintext highlighter-rouge">yyyy-mm-dd</code>
format just doesn’t feel natural. Adding functionality requires writing a <a href="https://go.dev">Go</a>
plugin which again may not be for everyeone.</p>
<p>But what makes <a href="https://gohugo.io">Hugo</a> stand apart is this one word: <strong>SPEED</strong>. Speed in setting
it up - just download the binary and you are ready to go. Whether you are serving
locally to test, or publishing your site for production, completing everything in
less than a second is what I will call <strong>GOD SPEED</strong>.</p>
<p>However, if I were to move all my work, then I will need to change the date format
again to something <code class="language-plaintext highlighter-rouge">Hugo</code> can undertand. Wish there was a simple way to define that
in the configuration.</p>
Revisiting the blog
https://sangupta.com/tech/revisiting-the-blog.html
2022-04-16T00:00:00+00:00
2022-04-16T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>It has been many years I wrote something worthy. A lot has changed since then. In these years
front-end development has become more advanced with technologies like <a href="https://reactjs.org">ReactJS</a>, <a href="https://vuejs.org">Vue</a>,
<a href="https://svelte.dev">Svelte</a> etc. Static sites need no longer confined to generate-time data and basic text
processing. They should be able to make use of <a href="https://developer.mozilla.org/en-US/docs/Web/Web_Components">web components</a> to better the user experience.</p>
<p>Today, as I look at this space it feels out-dated, pale and buggy. The headers and left-hand
pane are not truly responsive. The typography does not properly align. Colors need more work.
But these are least of my concerns. As a developer, I still have no true control over the way
I write (seldom) and it gets published. For example, why should all posts be in a single folder,
or in sub-folders based on date/time, or based on tag names?</p>
<p>With <a href="https://jekyllrb.com">Jekyll</a> setting up the site/blog or adding a new section within the existing one is
difficult. Posts can only be recognized and paginated from under the <code class="language-plaintext highlighter-rouge">_posts</code> folder. There
is no way to filter the posts. The software required to generate locally (ruby, bundler) etc
itself takes time to install. And then you run into version issues. It took me 30+ minutes today
get Jekyll running again. I believe we developers should do better here.</p>
<p>As I go over the latest <a href="https://jekyllrb.com/docs/">documentation</a> (<a href="https://github.com/jekyll/jekyll/releases/tag/v4.2.2">version 4.2.2</a>), I can’t find an easy way:</p>
<ul>
<li>to customize functionality using Javascript</li>
<li>no way to optimize/thumbnail images directly</li>
<li>pull HTTP data to merge into posts</li>
<li>creating automatic post excerpt</li>
<li>inline/donwload external images</li>
<li>support for <a href="https://velocity.apache.org">Velocity templates</a> (yes, I come from Java world)</li>
<li>customizing date format to be used</li>
<li>and more…</li>
</ul>
<p>I am sure there would be 3rd party plugins to support some of the above. However, I think this
is something that should be supported out of the box. Over the next couple of days, I will evaluate
the latest in static site generators. Have heard a lot about <a href="https://gohugo.io">Hugo</a>, <a href="https://gatsbyjs.org">Gatsby</a> and <a href="https://nextjs.org">Next</a>.
Time to find a worthy replacement for <a href="https://jekyllrb.com">Jekyll</a>.</p>
Javascript classes with `prototype`
https://sangupta.com/tech/javascript-classes-with-prototype.html
2017-08-18T00:00:00+00:00
2017-08-18T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p><a href="https://babeljs.io">BabelJS</a> has made it quite easy to write stateful <a href="https://reactjs.org">ReactJS</a> components by
helping transpile the <code class="language-plaintext highlighter-rouge">class</code> based components into <code class="language-plaintext highlighter-rouge">function</code>’s. Classes are still
second class citizens in the Javascript world, as they are part of <a href="https://262.ecma-international.org/6.0/">ECMA Script 6</a>
which hasn’t yet gained wide adoption.</p>
<p>Though, this post is not about <a href="https://reactjs.org">ReactJS</a> but about classes in general. In this
post let’s revisit how classes can be created in Javascript using <code class="language-plaintext highlighter-rouge">function</code>s. As a
bonus we will also see how to create a singleton using a <code class="language-plaintext highlighter-rouge">function</code>.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// this function serves as the constructor</span>
<span class="kd">function</span> <span class="nx">Fruit</span><span class="p">(</span><span class="nx">color</span><span class="p">)</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">color</span> <span class="o">=</span> <span class="nx">color</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// let's add a weight parameter to the class</span>
<span class="c1">// and a pair of getter and setter</span>
<span class="nx">Fruit</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="nx">setWeight</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">weight</span><span class="p">)</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">weight</span> <span class="o">=</span> <span class="nx">weight</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">Fruit</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="nx">getWeight</span> <span class="o">=</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">this</span><span class="p">.</span><span class="nx">weight</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This seems very simple as we first use a simple <code class="language-plaintext highlighter-rouge">function</code> as a constructor and then
its <code class="language-plaintext highlighter-rouge">prototype</code> to add new methods to it. However, if you would like to use inheritance
it becomes a little tricky syntax. Suppose, we want to create another class called
<code class="language-plaintext highlighter-rouge">Orange</code> that inherits from <code class="language-plaintext highlighter-rouge">Fruit</code>. This <code class="language-plaintext highlighter-rouge">Orange</code> has a constructor that takes another
attribute called <code class="language-plaintext highlighter-rouge">variety</code> along with <code class="language-plaintext highlighter-rouge">color</code>. Let’s see how this is accomplished.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// create another function that serves as a constructor</span>
<span class="kd">function</span> <span class="nx">Orange</span><span class="p">(</span><span class="nx">color</span><span class="p">,</span> <span class="nx">variety</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// as in classes, call the super with color</span>
<span class="nx">Fruit</span><span class="p">.</span><span class="nx">call</span><span class="p">(</span><span class="k">this</span><span class="p">,</span> <span class="nx">color</span><span class="p">);</span>
<span class="c1">// assign remaining attributes</span>
<span class="k">this</span><span class="p">.</span><span class="nx">variety</span> <span class="o">=</span> <span class="nx">variety</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>So far, so good. However, we still need to set the prorotype chain to ensure that when
an <code class="language-plaintext highlighter-rouge">Orange</code> is created, the methods from <code class="language-plaintext highlighter-rouge">Fruit</code> are inherited as well. For the same, we
need to set the following on <code class="language-plaintext highlighter-rouge">Orange</code>:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nx">Orange</span><span class="p">.</span><span class="nx">prototype</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Fruit</span><span class="p">;</span>
<span class="nx">Orange</span><span class="p">.</span><span class="nx">prototype</span><span class="p">.</span><span class="kd">constructor</span> <span class="o">=</span> <span class="nx">Orange</span><span class="p">;</span>
</code></pre></div></div>
<p>And we are all set. Now if we create an <code class="language-plaintext highlighter-rouge">Orange</code> and call <code class="language-plaintext highlighter-rouge">setWeight()</code> method, it shall
work flawlessly.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// create a new instance</span>
<span class="kd">const</span> <span class="nx">or</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Orange</span><span class="p">(</span><span class="dl">'</span><span class="s1">reddish-orange</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">grapefruit</span><span class="dl">'</span><span class="p">);</span>
<span class="c1">// call any method from Fruit or Orange</span>
<span class="nx">or</span><span class="p">.</span><span class="nx">setWeight</span><span class="p">(</span><span class="mi">100</span><span class="p">);</span> <span class="c1">// works like a charm</span>
<span class="nx">or</span><span class="p">.</span><span class="nx">getWeight</span><span class="p">();</span> <span class="c1">// returns 100</span>
</code></pre></div></div>
<p>Now, let’s look at how to create a singleton.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">Apple</span> <span class="o">=</span> <span class="k">new</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="k">this</span><span class="p">.</span><span class="nx">type</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">green apple</span><span class="dl">"</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">color</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">red</span><span class="dl">"</span><span class="p">;</span>
<span class="k">this</span><span class="p">.</span><span class="nx">printAsString</span> <span class="o">=</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">this</span><span class="p">.</span><span class="nx">color</span> <span class="o">+</span> <span class="dl">'</span><span class="s1"> </span><span class="dl">'</span> <span class="o">+</span> <span class="k">this</span><span class="p">.</span><span class="nx">type</span> <span class="o">+</span> <span class="dl">'</span><span class="s1"> apple</span><span class="dl">'</span><span class="p">;</span>
<span class="p">};</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Instead of just declaring a normal <code class="language-plaintext highlighter-rouge">function</code>, we define and invoke an anonymous <code class="language-plaintext highlighter-rouge">function</code>
using the <code class="language-plaintext highlighter-rouge">new</code> keyword. This way the constructor to the object is no longer available to
other code, and hence, they cannot initialize the same.</p>
<p>Happy hacking!</p>
Announcing AM library 1.0.0
https://sangupta.com/tech/announcing-am-1.0.0.html
2016-12-22T00:00:00+00:00
2016-12-22T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>I am happy to announce the launch of a new library called <strong>AM</strong> or <strong>Assert-Mocks
for unit-testing Java servlet API code</strong>. The library aims to make it a lot easier
and faster to test the servlet API code, including and not limited to testing of
servlets, servlet filters and tag libraries. It helps remove the need of using
<a href="https://github.com/mockito/mockito">Mockito</a> framework for setting up
expectations.</p>
<!-- break here -->
<h2 id="usage">Usage</h2>
<p>The following two examples should help you get started easily. More examples
are being added to the <a href="https://github.com/sangupta/am">Github:sangupta/am</a>
repository.</p>
<h2 id="testing-a-simple-jsp-tag">Testing a simple JSP tag</h2>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">AmTagLibTestHelper</span><span class="o">.</span><span class="na">testTagOutput</span><span class="o">(</span>
<span class="c1">// the class implementing custom tag</span>
<span class="nc">MyCustomJSPTag</span><span class="o">.</span><span class="na">class</span><span class="o">,</span>
<span class="c1">// the expected String response</span>
<span class="n">expectedOutputWrittenToJspWriter</span><span class="o">,</span>
<span class="c1">// set the values before invocation</span>
<span class="k">new</span> <span class="nc">GenericConsumer</span><span class="o"><</span><span class="nc">MyCustomJSPTag</span><span class="o">>()</span> <span class="o">{</span>
<span class="c1">// method argument of same type as the first param</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">consume</span><span class="o">(</span><span class="nc">MyCustomJSPTag</span> <span class="n">tag</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// set the properties of the tag</span>
<span class="n">tag</span><span class="o">.</span><span class="na">setProperty1</span><span class="o">(</span><span class="n">prop1</span><span class="o">);</span>
<span class="n">tag</span><span class="o">.</span><span class="na">setProperty2</span><span class="o">(</span><span class="n">prop2</span><span class="o">);</span>
<span class="c1">// and so on...</span>
<span class="c1">// return either true or false</span>
<span class="c1">// it does not matter in this case</span>
<span class="k">return</span> <span class="kc">true</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">);</span>
</code></pre></div></div>
<h2 id="testing-a-servlet-filter">Testing a Servlet Filter</h2>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// obtain an instance of the filter</span>
<span class="nc">MyCustomFilter</span> <span class="n">filter</span> <span class="o">=</span> <span class="nc">AmServletFilterTestHelper</span><span class="o">.</span><span class="na">getFilterForUnitTesting</span><span class="o">(</span><span class="nc">MyCustomFilter</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
<span class="c1">// create request and response objects as filter will need them</span>
<span class="c1">// the AmHttpServletRequest.getDefault() method returns a request for a server</span>
<span class="c1">// deployed on localhost on port 80, and being hit with same machine where</span>
<span class="c1">// the servlet context is `context` and the path is `/home.html`</span>
<span class="nc">AmHttpServletRequest</span> <span class="n">request</span> <span class="o">=</span> <span class="nc">AmHttpServletRequest</span><span class="o">.</span><span class="na">getDefault</span><span class="o">(</span><span class="s">"home.html"</span><span class="o">);</span>
<span class="c1">// the response object to which filter will write</span>
<span class="nc">AmHttpServletResponse</span> <span class="n">response</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">AmHttpServletResponse</span><span class="o">();</span>
<span class="c1">// filter invocation</span>
<span class="nc">AmServletFilterTestHelper</span><span class="o">.</span><span class="na">assertFilterChainInvoked</span><span class="o">(</span><span class="n">filter</span><span class="o">,</span> <span class="n">request</span><span class="o">,</span> <span class="n">response</span><span class="o">);</span>
<span class="c1">// assert what was set in response</span>
<span class="nc">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="mi">200</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">getStatusCode</span><span class="o">());</span>
<span class="nc">Assert</span><span class="o">.</span><span class="na">assertEquals</span><span class="o">(</span><span class="s">"myCustomHeaderValue"</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">getHeader</span><span class="o">(</span><span class="s">"X-Custom-Header"</span><span class="o">));</span>
</code></pre></div></div>
<h2 id="source--downloads">Source & Downloads</h2>
<p>The source code for the library is available at <a href="https://github.com/sangupta/am">Github:sangupta/am</a></p>
<p>The library is available at the following Maven coordinates:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><dependency></span>
<span class="nt"><groupId></span>com.sangupta<span class="nt"></groupId></span>
<span class="nt"><artifactId></span>am<span class="nt"></artifactId></span>
<span class="nt"><version></span>1.0.0<span class="nt"></version></span>
<span class="nt"></dependency></span>
</code></pre></div></div>
<p>Happy Hacking!</p>
Releasing jerry-core 3.0.0
https://sangupta.com/tech/jerry-core-3-release.html
2016-12-21T00:00:00+00:00
2016-12-21T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<p>I am happy to announce that exactly after an year of the last release, I have a
new major release for <strong>jerry-core</strong> library: version <strong>3.0.0</strong>. You may get hold
of the library from Maven Central with the following coordinates:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><dependency></span>
<span class="nt"><groupId></span>com.sangupta<span class="nt"></groupId></span>
<span class="nt"><artifactId></span>jerry-core<span class="nt"></artifactId></span>
<span class="nt"><version></span>3.0.0<span class="nt"></version></span>
<span class="nt"></dependency></span>
</code></pre></div></div>
<!-- break here -->
<p>Following are the major changes in this release:</p>
<ul>
<li>Minimum supported JDK version is now 1.7 than earlier 1.6</li>
<li>Upgraded the dependencies to the latest available versions</li>
<li>Fixed a critical bug in <code class="language-plaintext highlighter-rouge">Base62Encoder</code> where conflicting codes were being
generated - this impacts if numbers encoded were both positive and negative. And
thus this makes us bump up the major version - as the <code class="language-plaintext highlighter-rouge">Base62Encoder</code> is no longer
compatible with codes generated with any of the previous versions</li>
</ul>
<p>Other additions to the library include (and not limited to):</p>
<ul>
<li>Added <code class="language-plaintext highlighter-rouge">ResourceUtils</code> to read files from classpath including from packaged JARs</li>
<li>Added <code class="language-plaintext highlighter-rouge">StringArrayIterator</code> to iterate over a string array using <code class="language-plaintext highlighter-rouge">Iterator</code></li>
<li>Added <code class="language-plaintext highlighter-rouge">IndentedStringWriter</code> that takes care of writing long indented text that
breaks at a given line length</li>
<li>Added <code class="language-plaintext highlighter-rouge">count</code>, <code class="language-plaintext highlighter-rouge">removeAll</code>, <code class="language-plaintext highlighter-rouge">ltrim()</code> and <code class="language-plaintext highlighter-rouge">rtrim()</code> methods to <code class="language-plaintext highlighter-rouge">StringUtils</code></li>
<li>Updated <code class="language-plaintext highlighter-rouge">ReflectionUtils</code> to bind values to object-wrappers for primitives</li>
<li>Allow counter names to be read from <code class="language-plaintext highlighter-rouge">IntegerCounter</code> and <code class="language-plaintext highlighter-rouge">LongCounter</code></li>
<li>Added <code class="language-plaintext highlighter-rouge">SimpleMultiMap.numValues()</code></li>
<li>Added <code class="language-plaintext highlighter-rouge">StringUtils.wildcardMatch()</code> method</li>
<li>Added <code class="language-plaintext highlighter-rouge">jitpack.yml</code> for allowing <code class="language-plaintext highlighter-rouge">jerry-core</code> via https://jitpack.io</li>
<li>Added <code class="language-plaintext highlighter-rouge">isJDK8()</code> and <code class="language-plaintext highlighter-rouge">isJDK9()</code> methods to <code class="language-plaintext highlighter-rouge">JDKUtils</code></li>
<li>Added <code class="language-plaintext highlighter-rouge">asMap()</code> and <code class="language-plaintext highlighter-rouge">clear</code> methods to <code class="language-plaintext highlighter-rouge">IntegerCounter</code> and <code class="language-plaintext highlighter-rouge">LongCounter</code></li>
<li>Added <code class="language-plaintext highlighter-rouge">getNextSetBit()</code> method to <code class="language-plaintext highlighter-rouge">BitArray</code> and implementations</li>
<li>Added <code class="language-plaintext highlighter-rouge">AdvancedStringReader</code> and corresponding unit-tests</li>
</ul>
<p>And lastly, the <strong>Javadocs</strong> have been updated heavily to add missing documentation
and update the existing one to bring in more clarity.</p>
<p>Usage examples will be posted soon to this blog :)</p>
<p>Hope you find this release useful.</p>
Fastest way to merge multiple integer sets
https://sangupta.com/tech/merge-multiple-integer-sets.html
2016-09-18T00:00:00+00:00
2016-09-18T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<h1 id="problem">Problem</h1>
<p>You are given multiple integer sets and you need to merge them into a single set in the fastest possible
way.</p>
<h1 id="solution">Solution</h1>
<div class="alert alert-info">
The always updated article is always available on
<a href="https://github.com/sangupta/ps/blob/master/solutions/2016/fastest-duplicates-integer-sets.md">Github:sangupta/ps</a> repo
</div>
<p>Suppose we have 3 arrays of integer sets and we need to merge them. The fastest solution would be possible
in time of <code class="language-plaintext highlighter-rouge">O(n1 + n2 + n3)</code> where <code class="language-plaintext highlighter-rouge">n1</code>, <code class="language-plaintext highlighter-rouge">n2</code>, <code class="language-plaintext highlighter-rouge">n3</code> are the lengths of the three sets respectively. The solution
lies in using a bit-vector (also called as bit-set or a bit-array) to represent elements using the index
and then iterating over them.</p>
<ul>
<li>Construct a bit-array</li>
<li>Iterate over the first array and for each integer set the corresponding bit to true</li>
<li>Repeat the above process for remaining arrays</li>
<li>Any duplicate element will result in setting <code class="language-plaintext highlighter-rouge">true</code> an already set bit</li>
<li>The resultant bit-array is the merge of all the arrays</li>
</ul>
<!-- break here -->
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">mergeArrays</span><span class="o">(</span><span class="kt">int</span><span class="o">[]</span> <span class="n">array1</span><span class="o">,</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">array2</span><span class="o">,</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">array3</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">BitSet</span> <span class="n">bitSet</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BitSet</span><span class="o">();</span>
<span class="c1">// start merging</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">num</span> <span class="o">:</span> <span class="n">array1</span><span class="o">)</span> <span class="o">{</span>
<span class="n">bitSet</span><span class="o">.</span><span class="na">setBit</span><span class="o">(</span><span class="n">num</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">num</span> <span class="o">:</span> <span class="n">array2</span><span class="o">)</span> <span class="o">{</span>
<span class="n">bitSet</span><span class="o">.</span><span class="na">setBit</span><span class="o">(</span><span class="n">num</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">num</span> <span class="o">:</span> <span class="n">array3</span><span class="o">)</span> <span class="o">{</span>
<span class="n">bitSet</span><span class="o">.</span><span class="na">setBit</span><span class="o">(</span><span class="n">num</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
<span class="c1">// output</span>
<span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span><span class="o">;</span>
<span class="k">do</span> <span class="o">{</span>
<span class="n">index</span> <span class="o">=</span> <span class="n">bitSet</span><span class="o">.</span><span class="na">getNextSetBit</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">index</span> <span class="o"><</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="o">}</span> <span class="k">while</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The solution above can be extended to as many arrays as are provided in the problem definition. The time to
sort will still remain <code class="language-plaintext highlighter-rouge">O(N)</code> where <code class="language-plaintext highlighter-rouge">N</code> is the sum of total number of elements across all provided arrays.</p>
<h2 id="optimizations-available">Optimizations available</h2>
<ul>
<li>
<p>One can make use of sparsed-bit-arrays to reduce memory consumption. Refer to [brettwooldridge/SparseBitSet]
(https://github.com/brettwooldridge/SparseBitSet) for one such implementation.</p>
</li>
<li>
<p>If the arrays are really, really huge - an implementation that uses file-based persistence of a bit-array
can be used. Refer to <a href="https://github.com/sangupta/jerry-core/blob/master/src/main/java/com/sangupta/jerry/ds/bitarray/MMapFileBackedBitArray.java">one such implementation</a>
available in the <a href="https://github.com/sangupta/jerry-core">jerry-core</a> project.</p>
</li>
</ul>
Fastest sorting of integers
https://sangupta.com/tech/fastest-sorting-of-integers.html
2016-09-17T00:00:00+00:00
2016-09-17T00:00:00+00:00
Sandeep Gupta
https://sangupta.com
<h1 id="problem">Problem</h1>
<p>We are given a huge array of bounded, non-duplicate, positive integers that need
to be sorted. What’s the fastest way to do it.</p>
<h1 id="solution">Solution</h1>
<div class="alert alert-info">
The always updated article is always available on
<a href="https://github.com/sangupta/ps/blob/master/solutions/2016/fastest-sorting-integers.md">Github:sangupta/ps</a> repo
</div>
<p>Most of the interview candidates that I have talked to about this problem come up
with the quick answer as <code class="language-plaintext highlighter-rouge">MergeSort</code> or divide-and-conquer. The cost of sorting
being <code class="language-plaintext highlighter-rouge">O(N * Log(N))</code> - which in this case is not the fastest sorting time.</p>
<p>The fastest time to sort an integer array is <code class="language-plaintext highlighter-rouge">O(N)</code>. Let me explain how.</p>
<ul>
<li>Construct a boolean array of length N</li>
<li>For every integer <code class="language-plaintext highlighter-rouge">n</code> in array, mark the boolean at index <code class="language-plaintext highlighter-rouge">n</code> in array as <code class="language-plaintext highlighter-rouge">true</code></li>
<li>Once the array iteration is complete, just iterate over the boolean array again
to print all the indices that have the value set as <code class="language-plaintext highlighter-rouge">true</code></li>
</ul>
<!-- break here -->
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">sortBoundedIntegers</span><span class="o">(</span><span class="kt">int</span><span class="o">[]</span> <span class="n">array</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">/// the regular checks first</span>
<span class="k">if</span><span class="o">(</span><span class="n">array</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span><span class="o">;</span>
<span class="o">}</span>
<span class="k">if</span><span class="o">(</span><span class="n">array</span><span class="o">.</span><span class="na">length</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">// build the boolean result array</span>
<span class="kt">boolean</span><span class="o">[]</span> <span class="n">result</span> <span class="o">=</span> <span class="k">new</span> <span class="kt">boolean</span><span class="o">[</span><span class="n">array</span><span class="o">.</span><span class="na">length</span><span class="o">];</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">index</span> <span class="o"><</span> <span class="n">array</span><span class="o">.</span><span class="na">length</span><span class="o">;</span> <span class="n">index</span><span class="o">++)</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">num</span> <span class="o">=</span> <span class="n">array</span><span class="o">[</span><span class="n">index</span><span class="o">];</span>
<span class="n">result</span><span class="o">[</span><span class="n">num</span><span class="o">]</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
<span class="o">}</span>
<span class="c1">// print the result</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">index</span> <span class="o"><</span> <span class="n">result</span><span class="o">.</span><span class="na">length</span><span class="o">;</span> <span class="n">index</span><span class="o">++)</span> <span class="o">{</span>
<span class="k">if</span><span class="o">(</span><span class="n">result</span><span class="o">[</span><span class="n">index</span><span class="o">])</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<h2 id="additional-constraints-and-optimizations-available">Additional constraints and optimizations available</h2>
<ul>
<li>
<p>A <code class="language-plaintext highlighter-rouge">boolean</code> occupies one-byte of memory. Thus, switching to a bit-vector (also
called as bit-array) will reduce the memory consumption by a factor of 8. Check
code sample 2.</p>
</li>
<li>
<p>In case the integers are also negative, another bit-array can be used to check
for negatives, and then both iterated one-after-another to produce result. Check
code sample 2.</p>
</li>
<li>
<p>To further reduce the memory consumption, one can make use of sparsed-bit-arrays.
This can lead to huge drop in memory consumption if the integers are spaced apart
a lot. Check code sample 2. Refer to <a href="https://github.com/brettwooldridge/SparseBitSet">brettwooldridge/SparseBitSet</a>
for one such implementation.</p>
</li>
<li>
<p>In case the integer array contains duplicates, use a small <code class="language-plaintext highlighter-rouge">short</code> array than
the <code class="language-plaintext highlighter-rouge">boolean</code> array to hold the number of times an integer has been seen, thus
still sorting in <code class="language-plaintext highlighter-rouge">O(N)</code> time.</p>
</li>
</ul>
<h3 id="code-sample-2">Code Sample 2</h3>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">sortBoundedIntegers</span><span class="o">(</span><span class="kt">int</span><span class="o">[]</span> <span class="n">array</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// run regular checks</span>
<span class="kd">final</span> <span class="kt">int</span> <span class="n">len</span> <span class="o">=</span> <span class="n">array</span><span class="o">.</span><span class="na">length</span><span class="o">;</span>
<span class="c1">// considering SparsedBitSet is an implementation available</span>
<span class="nc">BitSet</span> <span class="n">negative</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">SparsedBitSet</span><span class="o">(</span><span class="n">len</span><span class="o">);</span>
<span class="nc">BitSet</span> <span class="n">positive</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">SparsedBitSet</span><span class="o">(</span><span class="n">len</span><span class="o">);</span>
<span class="c1">// sort</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">index</span> <span class="o"><</span> <span class="n">len</span><span class="o">;</span> <span class="n">index</span><span class="o">++)</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">num</span> <span class="o">=</span> <span class="n">array</span><span class="o">[</span><span class="n">index</span><span class="o">];</span>
<span class="k">if</span><span class="o">(</span><span class="n">num</span> <span class="o"><</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="n">num</span> <span class="o">=</span> <span class="mi">0</span> <span class="o">-</span> <span class="n">num</span><span class="o">;</span> <span class="c1">// convert to positive</span>
<span class="n">negative</span><span class="o">.</span><span class="na">setBit</span><span class="o">(</span><span class="n">num</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
<span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
<span class="n">positive</span><span class="o">.</span><span class="na">setBit</span><span class="o">(</span><span class="n">num</span><span class="o">,</span> <span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="c1">// output</span>
<span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span><span class="o">;</span>
<span class="k">do</span> <span class="o">{</span>
<span class="n">index</span> <span class="o">=</span> <span class="n">negative</span><span class="o">.</span><span class="na">getNextSetBit</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">index</span> <span class="o"><</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="mi">0</span> <span class="o">-</span> <span class="n">index</span><span class="o">);</span>
<span class="o">}</span> <span class="k">while</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
<span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span><span class="o">;</span>
<span class="k">do</span> <span class="o">{</span>
<span class="n">index</span> <span class="o">=</span> <span class="n">positive</span><span class="o">.</span><span class="na">getNextSetBit</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="k">if</span><span class="o">(</span><span class="n">index</span> <span class="o"><</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">index</span><span class="o">);</span>
<span class="o">}</span> <span class="k">while</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<h1 id="additional-reading">Additional Reading</h1>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Bit_array">Bit-Vectors</a></li>
<li><a href="https://en.wikipedia.org/wiki/Hash_function">Hashing</a></li>
<li><a href="https://en.wikipedia.org/wiki/Bloom_filter">Bloom Filters</a></li>
</ul>