<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Zero Brane</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/" />
    <link rel="self" type="application/atom+xml" href="http://notebook.kulchenko.com/atom.xml" />
    <id>tag:notebook.kulchenko.com,2009-04-10://2</id>
    <updated>2012-05-19T01:45:14Z</updated>
    <subtitle>By seeking, you will discover...</subtitle>
    <generator uri="http://www.sixapart.com/movabletype/">Movable Type 4.25</generator>

<entry>
    <title>Dynamic execution of embedded gists</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/dynamic-execution-of-embedded-gists" />
    <id>tag:notebook.kulchenko.com,2012://2.127</id>

    <published>2012-03-31T21:47:57Z</published>
    <updated>2012-05-19T01:45:14Z</updated>

    <summary>Gists are short code snippets that are useful for quick and simple code sharing. However, with all the functions provided by GitHub, gists are useful for much more than that: they are git repositories that have limited functions, but are...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="browser" label="browser" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p><a href="https://gist.github.com/">Gists</a> are short code snippets that are useful for quick and simple code sharing. However, with all the functions provided by GitHub, gists are useful for much more than that: they are git <em>repositories</em> that have limited functions, but are <em>versioned</em>, can be <em>forked</em>, can be <em>public/private</em>, and can have their code <em>embedded</em> in web pages. It is this last feature I will use to show how the code that is stored and managed in a gist can be executed on a page.</p>]]>
        <![CDATA[<p>First of all, embedding of a gist is very easy: you add <code>&lt;script src=&quot;https://gist.github.com/2278953.js&quot;&gt;</code> tag to your page; <code>2278953</code> here is the unique ID of a gist I'm using in this example. Each public gist has "embed" or "show embed" link that gives your the code to use in your page. You can also embed individual files (<code>&lt;script src=&quot;https://gist.github.com/2278953.js?file=01-init-canvas.lua&quot;&gt;&lt;/script&gt;</code>). If you access that <span class="caps">URL </span>you get back several <code>document.write</code> calls that have the content of your gist and the code to do styling and syntax highlighting.</p>

<p>For this example I <a href="https://gist.github.com/2278953">created a gist</a> that has two files with some Lua code that I want to execute in a browser. To execute this Lua code I use the <a href="/programming/drawing-on-browser-canvas-with-lua">mechanism I described earlier</a>.</p>

<p>Now, with the code of our gist available on the client, all we need to do is to write some JavaScript code to parse and execute the gist. This may look something like this:</p>

<pre><code>function luagist() {
  var gists = document.getElementsByClassName(&quot;gist-data&quot;);
  for (var i = 0; i &lt; gists.length; i++) {
    var text = [];
    var lines = gists[i].getElementsByClassName(&quot;line&quot;);
    for (var l = 0; l &lt; lines.length; l++)
      text.push(&quot;textContent&quot; in lines[l] ? lines[l].textContent : lines[l].innerText);
    (lua_load(text.join('\n')))();
  }
}</code></pre>

<p>The logic here is simple: GitHub returns elements with <code>gist-data</code> class (one per each file in your gist). One complication is that every file is split into multiple lines and then split into tokens to provide syntax highlighting. It can be quite cumbersome to assemble the original content from all these fragments, but fortunately <span class="caps">HTML </span>provides <code>textContent</code> and <code>innerText</code> methods that do all the heavy lifting. The script processes every line and then joins all the lines in one script that then gets interpreted and executed. This execution is done one per file, in the same order as your files are presented in a gist (hence you may want to use <code>01-</code>, <code>02-</code>, or similar prefixes to make sure the files are loaded in the order you need).</p>

<p>The last step, which you may not need, is to hide the content on the page; this is done with <code>.gist { display: none }</code>. You can see the <a href="/programming/lua-gist">complete demo here</a>. The disadvantage is that the access may be occasionally slow; the advantage is that any changes done to the "gisted" script become immediately available to the users.</p>

<p>Another way to embed the Lua code you may want to execute is to use a <code>&lt;script&gt;</code> tag with a different <code>type</code> specific for your scripts. I picked <code>text/x-luascript</code>; the script code may look similar to this:</p>

<pre><code>&lt;script type=&quot;text/x-luascript&quot;&gt;
local canvas = document:getElementById(&quot;canvas&quot;)
canvas.width = canvas.width -- clear the canvas
local ctx = canvas:getContext(&quot;2d&quot;)

ctx.fillStyle = &quot;rgb(200,0,0)&quot;
ctx:fillRect(130, 130, 50, 50)
ctx.fillStyle = &quot;rgba(0, 0, 200, 0.1)&quot;

for i = 1, 100 do
  ctx:fillRect(math.random(0, 300), math.random(0, 300), 50, 50)
end
&lt;/script&gt;</code></pre>

<p>The JavaScript code to process this content is only few lines long:</p>

<pre><code>function luascript() {
  var tags = document.getElementsByTagName(&quot;script&quot;);
  for (var i = 0; i &lt; tags.length; i++) {
    if (tags[i].type == &quot;text/x-luascript&quot;) (lua_load(tags[i].text))();
  }
}</code></pre>

<p>The code loops through all <code>&lt;script&gt;</code> elements, finds those that have the type we are interested in, and evaluates the content of those elements. You can see <a href="/programming/lua-script">this code in action here</a>.</p>]]>
    </content>
</entry>

<entry>
    <title>Lua: Good, bad, and ugly parts</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/lua-good-different-bad-and-ugly-parts" />
    <id>tag:notebook.kulchenko.com,2012://2.125</id>

    <published>2012-03-26T00:22:12Z</published>
    <updated>2012-05-20T03:39:13Z</updated>

    <summary>I have been programming in Lua for about 9 months and it seemed like a good time to pause and reflect on my experience with it. I have used various languages over the years -- Perl (soaplite.com, and other projects,...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I have been programming in Lua for about 9 months and it seemed like a good time to pause and reflect on my experience with it. I have used various languages over the years -- Perl (<a href="http://soaplite.com/">soaplite.com</a>, and other projects, including my current consulting work), C (<a href="http://notebook.kulchenko.com/dhcplite/dhcp-server-with-arduino-v012"><span class="caps">DHCPL</span>ite</a> and a <a href="http://notebook.kulchenko.com/juggling/robot-juggles-two-ping-pong-balls">ping-pong juggling robot</a>), JavaScript (<a href="http://notebook.kulchenko.com/maps/google-maps-using-multiple-api-versions">experiments with Google Maps and canvas</a>), <span class="caps">MATLAB </span>(<a href="http://notebook.kulchenko.com/juggling/robot-juggles-two-ping-pong-balls">ping-pong juggling robot</a>), and others, from Turbo Pascal to F# -- and it was interesting to see how Lua compares to the other languages I've worked with. I have done different types of projects in Lua: a remote debugger (<a href="https://github.com/pkulchenko/MobDebug">MobDebug</a>), extending a Lua <span class="caps">IDE </span>(<a href="https://github.com/pkulchenko/ZeroBraneStudio">ZeroBrane Studio</a>), a mobile app (<a href="https://github.com/pkulchenko/LuaRemote">LuaRemote</a>), several educational scripts (<a href="https://github.com/pkulchenko/ZeroBraneEduPack">EduPack</a>), and a demo of <a href="http://notebook.kulchenko.com/programming/drawing-on-browser-canvas-with-lua">drawing on browser canvas with Lua</a>.</p>

<p>I have come across several detailed lists that mention good and not-so-good parts of Lua (for example, <a href="http://lua-users.org/lists/lua-l/2007-11/msg00248.html">Lua benefits</a>, <a href="http://blog.datamules.com/blog/2012/01/30/why-lua/">why Lua</a>, <a href="http://lua-users.org/lists/lua-l/2012-01/msg00731.html">why Lua is not more widely used</a>, <a href="http://stackoverflow.com/questions/564665/advantages-of-lua">advantages of Lua</a>, <a href="http://gergely.imreh.net/blog/2011/07/language-of-the-month-lua-part-2/">Lua good/bad</a>, <a href="http://stackoverflow.com/questions/1022560/subtle-differences-between-javascript-and-lua">Lua vs. JavaScript</a>, and <a href="http://www.luafaq.org/gotchas.html">Lua Gotchas</a>), but I found that some of the features that tripped me or that I cared about were not listed, so I put together my own list. It is far from being comprehensive and some aspects of the language are not covered (for example, math and string libraries), but it captures the gist of my experience with the language.</p>]]>
        <![CDATA[<h2>Good</h2>


<ul>
<li>Small: <a href="https://github.com/LuaDist/lua">20000 lines of C code</a> that can be built into a 182K executable interpreter (under Linux).</li>
<li>Portable: builds on any platform with an <span class="caps">ANSI</span> C compiler. You can see it running on almost anything from <a href="http://www.eluaproject.net/">microcontrollers</a> and <a href="http://hempeldesigngroup.com/lego/pblua/">Lego Minstorms <span class="caps">NXT</span></a>, to <a href="https://github.com/divineprog/mobilelua">mobile platforms</a>, to <a href="http://blog.brokenfunction.com/2011/11/how-to-make-a-massively-cross-platform-game/">game consoles</a>, to a <a href="http://notebook.kulchenko.com/programming/drawing-on-browser-canvas-with-lua">browser</a> (translated to JavaScript).</li>
<li><a href="http://www.lua.org/pil/24.html">Embedded and extensible language</a> that provides a straightforward interface to/from C/C++.</li>
<li>Sufficiently fast: <a href="http://shootout.alioth.debian.org/">performs well comparing to other languages</a> and has a <a href="http://luajit.org/performance.html"><span class="caps">JIT </span>compiler</a> that noticeably improves performance on many tasks; those who may still be not satisfied with the performance, can implement critical parts in C and, given the ease of integration with C, still benefit from other good aspects.</li>
<li>Well documented: <a href="http://www.lua.org/manual/">reference manual</a>, <a href="http://lua.org/pil/">book</a>, <a href="http://lua-users.org/wiki/">wiki</a>, <a href="http://www.capgo.com/Resources/SoftwareDev/LuaShortRef51.pdf">6-page short reference</a> and more.</li>
<li>Friendly and enthusiastic <a href="http://lua-users.org/">community</a>. Between the excellent documentation, the wiki, the <a href="http://www.lua.org/lua-l.html">mailing list</a>, and <a href="http://stackoverflow.com/questions/tagged/lua">StackOverflow</a>, I didn't have any issues finding answers to my questions.</li>
<li>Clean and simple syntax suitable for beginners and accessible to non-programmers. Lua has borrowed most of its control syntax from <a href="http://en.wikipedia.org/wiki/Modula">Modula</a>, the descendent of <a href="http://en.wikipedia.org/wiki/Pascal_%28programming_language%29">Pascal</a>, which was widely used in education as an introductory language. I still remember using early versions of Philippe Kahn's fast and elegant <a href="http://en.wikipedia.org/wiki/Turbo_Pascal">Turbo Pascal</a> <span class="caps">IDE.</span></li>
<li>Integrated interpreter: just run <code>lua</code> from the command line.</li>
<li>Native support for <a href="http://www.lua.org/pil/9.html">coroutines</a> to implement <a href="http://www.lua.org/pil/9.3.html">iterators</a> and <a href="http://www.lua.org/pil/9.4.html">non-preemptive multi-threading</a>.</li>
<li>Incremental <a href="http://www.lua.org/manual/5.1/manual.html#2.10">garbage collector</a> that has low latency, no additional memory cost, little implementation complexity, and support for <a href="http://www.lua.org/pil/17.html">weak tables</a>.</li>
<li>Powerful heterogeneous <a href="http://www.lua.org/pil/2.5.html">tables</a> that store values of any type (except <code>nil</code>) and can be indexed by values of any type (except <code>nil</code>): <code>[1, 2, 5, foo = &quot;bar&quot;, [func] = &quot;something&quot;, [&quot;some spaces&quot;] = value()]</code>.</li>
<li>Lexical scoping.</li>
<li><a href="http://www.lua.org/pil/6.html">Functional programming</a> with <a href="http://www.lua.org/pil/5.html">first class functions</a> and <a href="http://www.lua.org/pil/6.1.html">closures</a>.</li>
<li><a href="http://www.lua.org/pil/6.3.html">Tail calls</a>: <code>return functioncall()</code>.</li>
<li>Recursive functions don't need to be pre-declared: <code>local function foo() ... foo() ... end</code>; note this doesn't work with <code>local foo = function() ... foo() ... end</code>.</li>
<li>Functions return <a href="http://www.lua.org/pil/5.1.html">multiple values</a>: <code>return 1, 2, 3</code>. The caller can expect any number of values returned: if less than three is expected, the rest is discarded and if more than three is expected, the rest is <code>nil</code>-initialized.</li>
<li>Functions allow variable number of parameters with <code>function foo(...) local args = {...}; bar(param, ...) end</code>.</li>
<li>Tables can be "unpacked" into a list of parameters with <code>unpack</code> (or <code>table.unpack</code> in Lua 5.2): <code>print(unpack({1, 2, 3}))</code> prints <code>1 2 3</code>.</li>
<li>Manipulating environments (<code>getfenv</code> and <code>setfenv</code> in Lua 5.1 and <code>_ENV</code> manipulation in Lua 5.2), which allows building <a href="http://lua-users.org/wiki/SandBoxes">sandboxes</a> among other things.</li>
<li>Assignment to a list of variables: <code>local a, b, c = 1, 2</code>, <code>x, y = y, x</code>, or <code>a, b = foo()</code>.</li>
<li>Multiline strings (using <code>[[...]]</code>; can be enclosed with <code>[[...[=[...]=]...]]</code>) and comments (<code>--[[...]]</code>).</li>
<li>Semicolon as a statement separator is optional (mostly used to resolve ambiguous cases as in <code>a = f; (g).x(a)</code>).</li>
<li>Overloading using <a href="http://www.lua.org/manual/5.1/manual.html#2.8">metatables</a>.</li>
<li><a href="http://metalua.luaforge.net/">Metaprogramming</a> to do things from getting and modifying an abstract syntax tree to creating a new syntax for your <a href="http://en.wikipedia.org/wiki/Domain-specific_language"><span class="caps">DSL</span></a>.</li>
<li>The <code>for</code> statement has two forms: <a href="http://www.lua.org/pil/4.3.5.html">generic</a> (loops over iterators: <code>for a in iter() do ... end</code>) and <a href="http://www.lua.org/pil/4.3.4.html">numeric</a> (loops over numbers: <code>for a = 1, 5, 0.1 do ... end</code>); the numeric one supports all numeric values for steps (not just integers).</li>
<li>Syntactic sugar for function calls (<code>f'string'</code>, <code>f&quot;string&quot;</code>, <code>f[[string]]</code>, and <code>f{table}</code>) and method calls (<code>obj:m()</code>).</li>
<li>Simple yet powerful <a href="http://www.lua.org/pil/23.html">debug</a> library.</li>
</ul>



<h2>Different</h2>


<ul>
<li>Tables and strings are indexed from 1 rather than 0.</li>
<li>Assigning <code>nil</code> as a value removes the element from a table. This is consistent with returning <code>nil</code> for non-existing element, so it makes no difference whether the element does not exist or exists with a value of <code>nil</code>. <code>a = {b = nil}</code> produces an empty table.</li>
<li>No integers as a separate numeric type; <a href="http://www.lua.org/pil/2.3.html">the number type</a> represent real numbers.</li>
<li>No classes; <a href="http://www.lua.org/pil/16.html">object-orientation</a> is implemented using <a href="http://www.lua.org/pil/2.5.html">tables</a> and <a href="http://www.lua.org/pil/2.6.html">functions</a>; inheritance is implemented using the <a href="http://www.lua.org/manual/5.1/manual.html#2.8">metatable</a> mechanism.</li>
<li>Method calls are implemented using <code>object:method(args)</code> notation, which is the same as <code>object.method(object, args)</code> notation, but with <code>object</code> evaluated only once.</li>
<li><code>nil</code> and <code>false</code> are the only false values; 0, 0.0, "0" and all other values evaluate as <code>true</code>.</li>
<li>Non-equality operator is ~= (for example, <code>if a ~= 1 then ... end</code>).</li>
<li><code>not, or, and</code> keywords used for logical operators.</li>
<li><a href="http://www.lua.org/pil/4.1.html">Assignments are statements</a>, which means there is no <code>a=b=1</code> or <code>if (a=1) then ... end</code>.</li>
<li>No <code>a+=1</code>, <code>a++</code>, or similar shorthand forms.</li>
<li>No <code>continue</code> statement, although there is an <a href="http://www.luafaq.org/#T1.26">explanation</a> and a number of alternatives, like using <code>repeat break until true</code> inside the loop to break out of or a <code>goto</code> statement introduced in Lua 5.2.</li>
<li>No <code>switch</code> statement.</li>
<li>Brackets may be required in some contexts; for example, <code>a = {}; a.field</code> works, but <code>{}.field</code> doesn't; the latter needs to be specified as <code>({}).field</code>.</li>
<li>A control variable in a loop is localized by default and is not available after the loop.</li>
<li>Limit and step values in the numeric <code>for</code> loop are <a href="http://www.lua.org/pil/4.3.4.html">cached</a>; this means that in <code>for i = init(), limit(), step() do ... end</code> all three functions <code>init</code>, <code>limit</code>, and <code>step</code> are called once before the loop is executed.</li>
<li><a href="http://www.lua.org/pil/4.3.1.html">Conditionals</a> and other control structures require no brackets.</li>
<li>Strings and numbers are automatically converted (if string is used where a number is expected and vice versa), but not in equality/inequality comparisons: <code>0 == &quot;0&quot;</code> is <code>false</code>, <code>{} ~= 1</code> is <code>true</code>, and <code>foo[&quot;0&quot;]</code> and <code>foo[0]</code> refer to two different keys in the table; other relational operators generate errors on comparing values of different types.</li>
<li>Both commas and <a href="http://www.lua.org/pil/3.6.html">semicolons</a> can be used in table literals; both can be placed before the closing curly bracket as an <a href="http://www.lua.org/pil/3.6.html">optional separator</a>: <code>a = {a = 1, b = 2, }</code>.</li>
<li>Smaller than expected number of components that are available "out of the box"; some people see this as "batteries not included". This is the other side of having a compact and portable core and is well compensated by having <a href="http://luarocks.org/">LuaRocks</a> and libraries like <a href="http://stevedonovan.github.com/Penlight/">Penlight</a>.</li>
</ul>



<h2>Bad</h2>


<ul>
<li>Limited <a href="http://www.lua.org/pil/8.4.html">error handling support</a> (using <a href="http://www.lua.org/manual/5.1/manual.html#pdf-pcall">pcall</a> and <a href="http://www.lua.org/manual/5.1/manual.html#pdf-xpcall">xpcall</a>), although some may <a href="http://lua-users.org/lists/lua-l/2010-09/msg00364.html">argue that it is sufficient</a> and just needs some syntactic sugar and more feature support (like deterministic finalizers). The combination of <code>pcall</code> and <code>error</code> is quite powerful, especially given that <code>error</code> can return anything (for example, a table) rather than just a string, but having <code>catch ... finally</code> constructs may be cleaner and simpler to read in many cases.</li>
<li>Global scoping by default (this has been <a href="http://lua-users.org/lists/lua-l/2010-02/msg00753.html">partially addressed in Lua 5.2</a>, which has no globals). There is a <a href="http://metalua.luaforge.net/src/lib/strict.lua.html">strict</a> module that requires all global variables to be initialized. I have not had many issues caused by uninitialized globals, but still put this one into the "bad" category as I once made a mistake of calling a variable "next" and not initializing it, which <a href="https://github.com/pkulchenko/ZeroBraneStudio/commit/867de8f3e46ea0ff868a9ae78c632a03fdf7747c">caused an issue</a> with an iterator in a completely different module as it overwrote the <a href="http://www.lua.org/pil/7.3.html">next</a> function used with iterators.</li>
<li>No <a href="http://lua-users.org/wiki/LuaUnicode">Unicode support</a> (at the very least you don't get <code>string.len</code> and pattern matching functions to recognize Unicode characters); there is a <a href="https://github.com/duncanc/icu4lua">binding</a> to <a href="http://en.wikipedia.org/wiki/International_Components_for_Unicode"><span class="caps">ICU</span></a> library that implements full Unicode support. See also this <a href="http://lua-users.org/lists/lua-l/2012-02/msg00241.html">message</a> and follow-ups for a good summary of what is already supported and what modifications may be required for string.* functions.</li>
<li>Limited <a href="http://www.lua.org/pil/20.1.html">pattern-matching support</a>, although the included one is still quite powerful. After using Perl for over 15 years, I miss some of the regexp features (mostly look-aheads, optional groups <code>(group )?</code>, and groups inside groups), but nothing that warrants the additional complexity in the implementation. Those who need more power in their regexps can use <a href="http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html">LPeg</a> and its <a href="http://www.inf.puc-rio.br/~roberto/lpeg/re.html">re</a> module.</li>
<li>No ternary operator; <a href="http://lua-users.org/wiki/TernaryOperator">several alternatives are available</a>. I usually end up using <code>foo = test and value1 or value2</code> form with the caveat that <code>value2</code> can be assigned if both <code>test</code> and <code>value1</code> end up being <code>false</code>.</li>
<li>No <span class="caps">POSIX </span>functions built-in. There is the <a href="https://github.com/rrthomas/luaposix">luaposix</a> module, but it requires compilation, which is not always an option. I didn't miss this much, but I did come across a case where I needed to get/set an environment variable, and having access to <code>getenv</code> and <code>setenv</code> would be convenient.</li>
<li>No class/object finalizers. Lua provides finalizer functionality through the  <a href="http://www.lua.org/pil/29.html">__gc metamethod</a>, but it is available only for userdata types (and not tables) and doesn't match the functionality provided by other languages, for example, <a href="http://perldoc.perl.org/perltoot.html#Destructors"><span class="caps">DESTROY</span></a> and <a href="http://perldoc.perl.org/perltoot.html#Class-Destructors"><span class="caps">END</span></a> methods in Perl.</li>
<li>No yielding between Lua and C code: <code>coroutine.yield</code> call across Lua/C boundary fails with <code>attempt to yield across metamethod/C-call boundary</code>. I happened to come across this error several times as I was doing async programming with <a href="http://w3.impa.br/~diego/software/luasocket/">luasocket</a> and coroutines, but solved it using the <a href="http://keplerproject.github.com/copas/">copas</a> module. This has been addressed in Lua 5.2.</li>
</ul>



<h2>Ugly</h2>

<ul>
<li>Number of elements in a table is not easy to get and the result depends on how you do this (or what you mean by "length"). This is probably not surprising, given how powerful tables are in Lua and the fact that they support flexible indexing (by numbers and any other Lua type except <code>nil</code>). Tables in Lua have two parts: an "array/vector" part (generated with <code>t = {1, 2, 3}</code>) and a "hash" part (generated with <code>t = {a = &quot;foo&quot;, [&quot;b&quot;] = 2}</code>); the two can be flexibly combined. <code>#table</code> returns the length of the shortest "array/vector" part (without any gaps) and <code>table.maxn(t)</code> returns the longest "array/vector" part (this function is removed in Lua 5.2). The "hash" part doesn't have a defined length. Both parts can be iterated over using the <code>pairs</code> method, which allows you to count the number of elements in them. However, <code>print(#{1, 2, nil, 3})</code> prints 4 and not 2 as one may expect, whereas <code>print(#{1, 2, nil, 3, nil})</code> prints 2. I'm sure there is a good reasonable explanation for this, but for now it is in the "ugly" bucket.</li>
<li><code>return</code> statement can't be used if it's not the last statement in a block; in other words, <code>function foo() print(1); return; print(2) end</code> will trigger an error <code>'end' expected...</code> or <code>unexpected symbol near &lt;whatever statement you have after 'return'&gt;</code> (depending on whether you have semicolon after <code>return</code> or not). Not that anyone would want use this for anything other than debugging, but I got bitten by it couple of times. I would have put this in the "different" category, but I find it inconsistent that I can't use <code>return</code>, but can use <code>do return end</code> in exactly the same place.

<strong>[Updated 5/19/2012]</strong> This also applies to <code>break</code> statement, although in Lua 5.2 <code>break</code> is <a href="http://lua-list.2524044.n2.nabble.com/patch-to-goto-optional-semicolon-changing-opcodes-td7566529.html">no longer required to be the last statement in a block</a>.</li>
<li>Only one value is returned from a function if it's not the last one in a list; for example:

<pre><code>  function f123() return 1, 2, 3 end
  function f456() return 4, 5, 6 end
  print(f123(), f456()) -- prints 1, 4, 5, 6
  print(f456(), f123()) -- prints 4, 1, 2, 3</code></pre>

The related behavior of <code>return</code> is also affected by this rule: <code>return f456()</code> returns three values, but <code>return (f456())</code> returns only one value (note the extra pair of parentheses). This matches the overall simplicity of the language and is <a href="http://www.lua.org/pil/5.1.html">well documented</a>, but I still find it to be a bit ugly (although ugliness as beauty is in the eye of the beholder).</li>
</ul>

<p>Overall, I have so far enjoyed the simplicity and consistency of the language, although there are few things I wish were done a bit differently. My eight-year-old son also picked Lua syntax quickly, which reminded me a lot about my experience with Turbo Pascal decades ago.</p>]]>
    </content>
</entry>

<entry>
    <title>Can Lua be faster than JavaScript in a browser?</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/can-lua-be-faster-than-javascript-in-a-browser" />
    <id>tag:notebook.kulchenko.com,2012://2.126</id>

    <published>2012-03-20T21:05:31Z</published>
    <updated>2012-05-19T01:50:20Z</updated>

    <summary>I have been working on getting Lua to run in a browser and was asked how much slower it is comparing to JavaScript. I put together two pages -- one for Lua and one for JavaScript -- to run the...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="browser" label="browser" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="canvas" label="canvas" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I have been working on <a href="/programming/drawing-on-browser-canvas-with-lua">getting Lua to run in a browser</a> and was asked how much slower it is comparing to JavaScript. I put together two pages -- one for <a href="/programming/lua-canvas">Lua</a> and one for <a href="/programming/js-canvas">JavaScript</a> -- to run the equivalent code to measure its performance and got a surprising result: Lua reliably outperformed JavaScript with the following code:</p>

<h3>Lua</h3>

<pre><code>local canvas = document:getElementById(&quot;canvas&quot;)
canvas.width = canvas.width -- clear the canvas
local ctx = canvas:getContext(&quot;2d&quot;)

ctx.fillStyle = &quot;rgb(200,0,0)&quot;
ctx:fillRect(130, 130, 50, 50)
ctx.fillStyle = &quot;rgba(0, 0, 200, 0.1)&quot;

for i = 1, 100 do
  ctx:fillRect(math.random(0, 300), math.random(0, 300), 50, 50)
end</code></pre>

<h3>JavaScript</h3>

<pre><code>var canvas = document.getElementById(&quot;canvas&quot;);  
canvas.width = canvas.width
var ctx = canvas.getContext(&quot;2d&quot;);
  
ctx.fillStyle = &quot;rgb(200,0,0)&quot;;
ctx.fillRect(130, 130, 50, 50);
ctx.fillStyle = &quot;rgba(0, 0, 200, 0.1)&quot;;

for (i = 1; i&lt;=100; i++) {
  ctx.fillRect(Math.random()*300, Math.random()*300, 50, 50);
}</code></pre>]]>
        <![CDATA[<p>This clearly wasn't something I expected as the Lua code compiles down to JavaScript, so it can be running with the same speed (in an ideal case), but clearly should not be running faster (I've averaged results over several runs, but you can see for yourself).</p>

<p>Even more puzzling was the fact that each individual part -- running a loop with <code>ctx.fillRect(0, 0, 50, 50)</code> or running the same loop with <code>Math.random()</code> call -- ran faster with the JavaScript code, but together they ran slower. It took a bit of experimenting, but in the end the mystery was solved: the difference was in <code>Math.random()*300</code> in JavaScript versus <code>math.random(0, 300)</code> in Lua.</p>

<p>As some of you might have noticed already, <code>math.random(0, 300)</code> call in Lua returns an integer value, whereas <code>Math.random()*300</code> call in JavaScript returns a float value. I was reading about canvas performance just a day earlier and there was a recommendation to <a href="http://www.html5rocks.com/en/tutorials/canvas/performance/#toc-avoid-float">avoid using floating point coordinates</a> as it turns on anti-aliasing, which significantly slows the drawing. That was it; when I added <code>Math.round()</code> call the performance returned to normal.</p>

<p>So, while I was at it, I decided to compare performance of Lua vs. JavaScript on some other calls. The code above was running 50-80% slower in Lua than in JavaScript (with 1000 iterations). A simple loop with one function call -- <code>for (i = 1; i&lt;=10000; i++) { Math.abs(-10); }</code> -- is 3-6 times faster in JavaScript. This is primarily because of table lookup that Lua version has to do. When I changed <code>for i = 1, 10000 do math.abs(-10) end</code> to <code>abs = math.abs; for i = 1, 10000 do abs(-10) end</code> it became only 2-3 times slower than the JavaScript code. Attribute assignment <code>foo.bar</code> had a similar performance: it was 3-6 times slower because of the table lookup and associated function calls in the Lua version. </p>

<p>Overall, I have been very satisfied with the difference in performance and will share the results when I get more complex code running.</p>]]>
    </content>
</entry>

<entry>
    <title>Drawing on browser canvas with Lua</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/drawing-on-browser-canvas-with-lua" />
    <id>tag:notebook.kulchenko.com,2012://2.123</id>

    <published>2012-03-16T17:13:24Z</published>
    <updated>2012-05-19T01:52:16Z</updated>

    <summary>As I was working on my turtle library that runs on a desktop, I thought it would be useful to have a way to execute the same Lua code in a browser (as a way to demonstrate/share the result). There...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="browser" label="browser" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="canvas" label="canvas" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>As I was working on <a href="/education/drawing-trees-with-turtles">my turtle library</a> that runs on a desktop, I thought it would be useful to have a way to execute the same Lua code in a browser (as a way to demonstrate/share the result). There are <a href="http://stackoverflow.com/questions/176235/are-there-any-recent-lua-to-javascript-converters-or-interpreters-somewhere">several options</a> to get Lua to run in a browser, but most of them can be placed in one of two categories: re-writing Lua VM in JavaScript (like <a href="http://emscripten.org/">emscripten</a>) and translating Lua into JavaScript (like <a href="https://github.com/mherkender/lua.js">lua.js</a>). There is also an option of writing a Lua VM in some other language, but this would require a plugin to be installed, with all associated hassles. Lua VM approach provides an option of full compatibility (as you would be running a regular Lua <span class="caps">VM, </span>just implemented in JavaScript), but tends to be slow. Translation to Lua gives you more speed (as the browser executes the JavaScript code of your application rather than VM) at the cost of not having Lua features that don't exist in JavaScript (like non-string keys in tables, coroutines, and some other things). As I was looking for a way to run simple scripts that work with graphics, the speed was more important than a full set of Lua features, so I decided to try <a href="https://github.com/mherkender/lua.js">lua.js</a>.</p>]]>
        <![CDATA[<p>I wanted to be able to run a simple Lua script that I directly mapped from a <a href="https://developer.mozilla.org/en/Drawing_Graphics_with_Canvas">JavaScript sample</a>:</p>

<pre><code> local canvas = document:getElementById(&quot;canvas&quot;);
 local ctx = canvas:getContext(&quot;2d&quot;);

 ctx.fillStyle = &quot;rgb(200,0,0)&quot;;
 ctx:fillRect(10, 10, 55, 50);

 ctx.fillStyle = &quot;rgba(0, 0, 200, 0.5)&quot;;
 ctx:fillRect(30, 30, 55, 50);</code></pre>

<p>It took only 5 minutes to get a simple script to execute (as in get-to-the-first-line), but I immediately run into a problem not knowing how to get <code>document.getElementById(&quot;canvas&quot;)</code> working correctly. The problem was that lua.js compiled this to a code that was trying to find <code>document</code> in its global table, where it obviously didn't exist. This was easy to fix: I added <code>lua_tableset(lua_script, &quot;document&quot;, document);</code> to my javascript code to set the <code>document</code> key to reference the real document table.</p>

<p>The next issue was that the code was trying (reasonably enough) to get <code>getElementById</code> from document using Lua methods, and it wasn't working as it was searching in pseudo tables that lua.js setup, rather than in the actual <code>document</code> table. To fix this, I added a check to <code>lua_rawget</code> method to work with "native" values:</p>

<pre><code>function lua_rawget(table, key) {
 if (typeof table == &quot;object&quot; &amp;&amp; table[key]) {
   return table[key];
 }</code></pre>

<p>The next problem was more interesting. The code correctly found the right method to execute, but failed to execute it with infamous <code>&quot;Could not convert JavaScript argument&quot; nsresult: &quot;0x80570009 (NS_ERROR_XPC_BAD_CONVERT_JS)&quot;</code> error. The interesting part was that after <code>func = document.getElementById</code>, <code>document.getElementById(&quot;canvas&quot;)</code> was working, while <code>func(&quot;canvas&quot;)</code> triggered that error. Even <code>func.apply(null, [&quot;canvas&quot;])</code>, which was the code that lua.js generated, produced the same result. It took a bit of googling and reading to realize what was going on. It turned out that <code>document.getElementById</code> call was passing <code>document</code> as the scope for the function, which made it all work. After I did the same thing with <code>func.apply(document, [&quot;canvas&quot;])</code>, this call started to work.</p>

<p>The next thing it failed on was the second call, <code>canvas:getContext(&quot;2d&quot;);</code>. The error was different this time -- <code>&quot;Illegal operation on WrappedNative prototype object&quot; nsresult: &quot;0x8057000c (NS_ERROR_XPC_BAD_OP_ON_WN_PROTO)&quot;</code> -- but now I was well equipped to deal with it. In this case it expected a different scope (canvas element, rather than document), so I did a proper fix to provide that context. I changed <code>lua_rawcall</code> to take on scope parameter, and changed <code>lua_mcall</code> to pass that parameter when it detected a "native" call:</p>

<pre><code>function lua_rawcall(func, args, scope) {
 try {
   return func.apply(scope, args);
 } catch (e) {</code></pre>

<pre><code>function lua_mcall(obj, methodname, args) {
 if (typeof obj == "object" && typeof obj[methodname] == &quot;function&quot;) {
   //  this is a JavaScript call, like document:getElementById
   return [lua_rawcall(obj[methodname], args, obj)];
 }
 else {
   return lua_call(lua_tableget(obj, methodname), [obj].concat(args));
 }
}</code></pre>

<p>The next issue was to get <code>ctx.fillStyle = &quot;rgb(200,0,0)&quot;</code> to work as lua.js was trying to set <code>ctx.fillStyle</code> in Lua tables, whereas I needed it to be set in JavaScript. I modified <code>lua_rawset</code> function in the same way I modified <code>lua_rawget</code>:</p>

<pre><code>function lua_rawset(table, key, value) {
 if (typeof table == &quot;object&quot; &amp;&amp; table[key]) {
   table[key] = value;
   return;
 }</code></pre>

<p>The last issue was that all Lua calls return a table (to allow multiple parameters to be returned), but native JS calls expect simple result; I modified the returned result in <code>lua_mcall</code> to be an array <code>return [...];</code> and things start to fly.</p>

<p>One last tweak I did was that instead of putting <code>lua_tableset(lua_script, &quot;document&quot;, document);</code> in the JavaScript code manually, I added this assignment to the code that lua2js and lua+parser.js generate (at the end of <code>case 1:</code> output):</p>

<pre><code>      &quot;G.str['document'] = document;\n&quot; +
      &quot;G.str['window'] = window;\n&quot; +</code></pre>

<p>You can play with the result here: <a href="/programming/lua-canvas">drawing on browser canvas with Lua demo</a>. Kudos to <a href="http://blog.brokenfunction.com/">Maximilian Herkender</a> for doing all the work that enabled this.</p>]]>
    </content>
</entry>

<entry>
    <title>Storing WiFi configuration in EEPROM on Arduino</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/embedded/storing-wifi-configuration-in-eeprom-on-arduino" />
    <id>tag:notebook.kulchenko.com,2012://2.124</id>

    <published>2012-03-15T02:22:41Z</published>
    <updated>2012-05-19T01:55:23Z</updated>

    <summary>The Arduino streaming using AJAX example provides a simple way to access analog and digital ports on Arduino over WiFi, but one of its deficiencies is the challenge of configuring WiFi. It establishes its own network, but is not able...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="embedded" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="ajax" label="ajax" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="arduino" label="arduino" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcplite" label="dhcplite" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>The <a href="/embedded/streaming-real-time-data-from-arduino-using-ajax-and-persistent-connections">Arduino streaming using <span class="caps">AJAX </span>example</a> provides a simple way to access analog and digital ports on Arduino over WiFi, but one of its deficiencies is the challenge of configuring WiFi. It establishes its own network, but is not able to connect to any of the existing network, which may be the configuration you prefer. It is easy to modify the code to make it connect to an access point, but the configuration would be hardcoded and any change would require a new firmware upload.</p>

<p>Since the example already provides a simple web interface, wouldn't it be convenient to have a way to update board's WiFi configuration from a browser and make the board to store and use that configuration? This is exactly what I am about to discuss.</p>]]>
        <![CDATA[<h2>Storing to and retrieving configuration from <span class="caps">EEPROM</span></h2>

<p>First, we need a way to store and retrieve data to and from <span class="caps">EEPROM. EEPROM </span>is the memory that keeps its values even when you remove any external power from the Arduino board. Ardino Uno (powered by ATmega328 processor) has 1KB of <span class="caps">EEPROM.</span> Arduino documentation has an example with the <a href="http://arduino.cc/playground/Code/EEPROMLoadAndSaveSettings">code to manage <span class="caps">EEPROM</span></a>. This is what I ended up with based on that code:</p>

<pre><code>#define CONFIG_VERSION &quot;ar1&quot;
#define CONFIG_START 0

struct WiFiStorageStruct {
  char version[4];
  char ssid[24];
  char pwd[16];
  byte addr[4];
  unsigned int id;
} WiFiConfig = {
  CONFIG_VERSION,
  &quot;NetworkConnectTo&quot;,
  &quot;&quot;,
  {0, 0, 0, 0},
  0
};

void loadConfig() {
  if (EEPROM.read(CONFIG_START + 0) == CONFIG_VERSION[0] &amp;&amp;
      EEPROM.read(CONFIG_START + 1) == CONFIG_VERSION[1] &amp;&amp;
      EEPROM.read(CONFIG_START + 2) == CONFIG_VERSION[2])
    for (unsigned int t=0; t&lt;sizeof(WiFiConfig); t++)
      *((char*)&amp;WiFiConfig + t) = EEPROM.read(CONFIG_START + t);
}

void saveConfig() {
  for (unsigned int t=0; t&lt;sizeof(WiFiConfig); t++)
    EEPROM.write(CONFIG_START + t, *((char*)&amp;WiFiConfig + t));
}</code></pre>

<p><code>version</code> field is used to identify the format version, <code>ssid</code>, <code>pwd</code>, and ip store your WiFi configuration, and id keeps unique id of the Arduino you are working with, which is useful when you need to have several instances running at the same time (as I do when I run several robots with the same firmware). You will see this id used in the network name, for example, <code>TestNetwork0</code>, <code>TestNetwork1</code>, and so on.</p>

<p>Note that both <code>loadConfig</code> and <code>saveConfig</code> simply read or write a specific number of char values starting at configured <code>CONFIG_START</code> position.</p>

<h2>Using stored configuration</h2>

<p>The logic is simple: the WiFi card attempts to connect to a configured network (if any) and if the connection fails, it will establish its own ad-hoc network. This way if you bring the card to a location where the configured network is not available, you can always access the card using its own network and configure it to use whatever access point you prefer. This is the code that implements this:</p>

<pre><code>   loadConfig();
    RedFly.scan();
    adhoc = ret = RedFly.join(WiFiConfig.ssid, WiFiConfig.pwd, INFRASTRUCTURE);
    if (ret) {
      char network[16];
      sprintf(network, &quot;TestNetwork%d&quot;, WiFiConfig.id);
      ret = RedFly.join(network, IBSS_CREATOR, 10);
    }
    if (!ret) {
      currentIP = adhoc ? serverIP : WiFiConfig.addr;
      ret = RedFly.begin(currentIP, gateway, netmask);
    }</code></pre>

<p>The <code>join</code> command returns a non-zero result when it fails. In this case it will try to establish its own network with a unique name (using the identifier stored with the configuration). There is no code that modifies this identifier as it is stored only once; you can do it with the following fragment: <code>loadConfig(); WiFiConfig.id = 1; saveConfig();</code>.</p>

<h2>Updating WiFi configuration</h2>

<p>Now we only need to implement a web interface to show the current configuration and process an updated on. We can update the main page with a link to the WiFi configuration form; the main page now looks like this:</p>

<p><a href="/images/arduino-wifi-1.png"><img src="/images/arduino-wifi-1.png" alt="" height="304" width="500" /></a></p>

<p>The configuration page is straightforward: its main component is the <code>&lt;form&gt;</code> that includes <span class="caps">SSID, </span>password and IP address fields that are stored as WiFi config. This page is returned as a response to <code>GET /wifi</code> request. Note that the strings are stored in <a href="http://arduino.cc/en/Reference/PROGMEM"><span class="caps">PROGMEM</span></a>, to reduce the amount of <span class="caps">SRAM </span>required for the application. These strings are still loaded in <span class="caps">SRAM </span>when they are used, but only temporarily and don't consume any <span class="caps">SRAM </span>if this branch of the code is not used.</p>

<p>The form doesn't specify any <span class="caps">URL, </span>only the method (POST) to be used to send the data with. The same <span class="caps">URL </span>(<code>..../wifi</code>) will be used in this case, which is exactly what we want.</p>

<pre><code>     const char *OK = PSTR(&quot;HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n&quot;);
     const char *STYLE = PSTR(&quot;\r\n&lt;html style='font-family:Verdana,Geneva,Georgia,Chicago,Arial,Sans-serif;color:#002d80'&gt;&quot;);
...
       else if (strncmp_P(out, PSTR(&quot;GET /wifi HTTP&quot;), 14) == 0) {
        RedFly.socketSendPGM(sock, OK);
        RedFly.socketSendPGM(sock, STYLE);
        RedFly.socketSendPGM(sock, PSTR(&quot;&lt;form method='post'&gt;Wifi configuration&lt;p&gt;SSID: &lt;input style='margin-left:48px' value='&quot;));
        RedFly.socketSend(sock, (uint8_t*)WiFiConfig.ssid, strlen(WiFiConfig.ssid));
        RedFly.socketSendPGM(sock, PSTR(&quot;' name='s'/&gt;&lt;br/&gt;Password: &lt;input type='password' name='p' style='margin-left:10px'/&gt;&lt;br/&gt;IP address: &lt;input name='a' value='&quot;));
        sprintf(out, &quot;%d.%d.%d.%d&quot;, WiFiConfig.addr[0], WiFiConfig.addr[1], WiFiConfig.addr[2], WiFiConfig.addr[3]);
        RedFly.socketSend(sock, (uint8_t*)out, strlen(out));
        RedFly.socketSendPGM(sock, PSTR(&quot;'/&gt; (xxx.xxx.xxx.xxx)&lt;br/&gt;&lt;br/&gt;Clicking &lt;input type='submit' value='Update'/&gt; will update the configuration and restart the board.&lt;/p&gt;&lt;/form&gt;&lt;/html&gt;&quot;));
        RedFly.socketClose(sock);
      }</code></pre>

<p>This is a snapshot of what you would see in the browser when you access an arduino board that runs this code:</p>

<p><a href="/images/arduino-wifi-2.png"><img src="/images/arduino-wifi-2.png" alt="" height="304" width="500" /></a></p>

<p>The next part is the processing of the <span class="caps">HTTP POST </span>request that includes the new configuration. The data will come in the body of <span class="caps">HTTP </span>request in the format: <code>s=SSID&amp;p=password&amp;a=1.2.3.4</code>. First we parse the request by starting from the last parameter and using sscanf to split the IP address into 4 numbers we are interested in. Then the password and the <span class="caps">SSID </span>get processed. The configuration is updated only when all three parameters are found in the request.</p>

<p>After the configuration is saved, the board is soft-restarted using a jump to zero address: <code>void(* reset) (void) = 0; reset();</code>. Note that while this may look like a restart, the controller itself is not reset and no hardware initialization is done. The board is restarted with the new configuration and if the configuration is correct, the board connects to the network.</p>

<pre><code>     else if (strncmp_P(out, PSTR(&quot;POST /wifi HTTP&quot;), 15)  0) {
        char *st, *fi;
        char *a = NULL, *p = NULL, *s = NULL;
        out[buf_len] = '\0';
        if (st = strstr(out, "a="))
          if (sscanf(st+2, "%d.%d.%d.%d", WiFiConfig.addr, WiFiConfig.addr+1, WiFiConfig.addr+2, WiFiConfig.addr+3)  4) a = st+2;
        if (st = strstr(out, &quot;p=&quot;))
          if (fi = strstr(st, &quot;&amp;&quot;)) { fi[0] = '\0'; p = st+2; }
        if (st = strstr(out, &quot;s=&quot;))
          if (fi = strstr(st, &quot;&amp;&quot;)) { fi[0] = '\0'; s = st+2; }

        RedFly.socketSendPGM(sock, OK);
        RedFly.socketSendPGM(sock, STYLE);
        if (a &amp;&amp; p &amp;&amp; s) {
          strcpy(WiFiConfig.pwd, p);
          strcpy(WiFiConfig.ssid, s);
          saveConfig();
          RedFly.socketSendPGM(sock, PSTR(&quot;Updated. The board has been restarted with the new configuration.&lt;/html&gt;&quot;));
          RedFly.socketClose(sock);

          // restart Arduino by calling a (pseudo)function at address 0
          void(* reset) (void) = 0; // declare reset function @ address 0
          reset();
        }
        else {
          RedFly.socketSendPGM(sock, PSTR(&quot;Error. Please return back and update the configuration.&lt;/html&gt;&quot;));
          RedFly.socketClose(sock);
        }
      }</code></pre>

<p>Last touch is that depending on whether there is any error and whether adhoc or AP connection is established this code will use an <span class="caps">LED </span>connected to pin 13 to signal its state:</p>

<pre><code>  // 10 blinks on error, 5 blinks on AP connection and 3 blinks on adhoc connection
  blink(LEDPIN, ret ? 10 : (adhoc ? 3 : 5));</code></pre>

<p>All the code is available in the <a href="https://github.com/pkulchenko/DHCPLite/blob/master/examples/DHCPLite/RedFly/RedFly.pde">updated <span class="caps">DHCPL</span>ite library</a>. Please leave a comment or send me an email if you notice something is not working as expected.</p>]]>
    </content>
</entry>

<entry>
    <title>DHCP Server with Arduino v0.12</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/dhcplite/dhcp-server-with-arduino-v012" />
    <id>tag:notebook.kulchenko.com,2012://2.122</id>

    <published>2012-03-10T18:05:31Z</published>
    <updated>2012-03-16T17:13:06Z</updated>

    <summary>This version of the library (v0.12) added support for Arduino 1.0. You can now use the same code with both Arduino 0.2x and Arduino 1.0. The previous update (v0.10) also fixed an issue with iOS clients and have been confirmed...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="dhcplite" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="arduino" label="arduino" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcp" label="dhcp" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcplite" label="dhcplite" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>This version of the <a href="https://github.com/pkulchenko/DHCPLite">library</a> (v0.12) added support for Arduino 1.0. You can now use the same code with both Arduino 0.2x and Arduino 1.0.</p>

<p>The previous update (v0.10) also fixed an issue with iOS clients and have been confirmed to work with iPhone (iOS version 2) and iPad (iOS version 5). If you still having troubles for getting it to work with iOS clients, please send me an email with details.</p>]]>
        
    </content>
</entry>

<entry>
    <title>What are people going to do for fun in 20 years?</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/education/what-are-people-going-to-do-for-fun-in-20-years-instead-of-what-they-do-now" />
    <id>tag:notebook.kulchenko.com,2012://2.120</id>

    <published>2012-01-21T00:29:03Z</published>
    <updated>2012-03-19T02:21:50Z</updated>

    <summary>In their last Request for Startups Y Combinator posed an interesting question: What are people going to do for fun in 20 years instead of what they do now? I asked my sixteen year old son and his idea was...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="education" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>In their last <a href="http://ycombinator.com/rfs9.html">Request for Startups</a> Y Combinator posed an interesting question: <em>What are people going to do for fun in 20 years instead of what they do now?</em> I asked <a href="http://daniil.kulchenko.com/">my sixteen year old son</a> and his idea was that in 20 years movies and TV are going to be much more interactive down to the ability to participate in the plot line and change it on the fly. According to him, there will be some sort of artificial intelligence that will be able to build a new story line, somewhat similar to the popular <a href="http://en.wikipedia.org/wiki/Choose_Your_Own_Adventure">Choose Your Own Adventure</a> book series. I think this is an interesting idea and may be definitely possible in 20 years, but something that doesn't really pull people away from screens.</p>

<p>My idea is different. I think with the current advances in miniaturization, robotics, open-source hardware, 3-d printing, connectivity, and mobile devices will lead to the ability of a large chunk of the population to <em>mass-produce things for fun</em>. This is in many ways similar to the role <a href="http://en.wikipedia.org/wiki/Artisan">artisans or craftsmen</a> played prior to the Industrial Revolution. The products they build will not be passive, like the vast majority of the currently produced items or tools is; they will be <em>interactive, entertaining, and practical</em>. In the same way as mobile devices enabled a market for mobile applications, in many cases simple and created by someone without much experience in the field, these advances will create <em>opportunities for people to design, prototype, and niche-produce items that other people will find interesting to interact with</em>. Niche-produce means producing 100-10,000 items, which is larger, than an individual craftsman can produce and smaller than mass production would bother with. These ideas are already being explored with Kickstarter-funded projects, using crowd-funding, which will also be leveraged by this new wave of artisans. I also believe that while connectivity and computers will play a critical role in this process, the focus of the activity will migrate back to the physical world and the items produced will take the form of physical goods more than virtual services or mobile applications.</p>]]>
        
    </content>
</entry>

<entry>
    <title>Drawing outside of OnPaint with wxWidgets</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/education/drawing-outside-of-onpaint-with-wxwidgets" />
    <id>tag:notebook.kulchenko.com,2012://2.119</id>

    <published>2012-01-21T00:17:55Z</published>
    <updated>2012-05-19T01:56:18Z</updated>

    <summary>As I have been working on the Turtle graphics library for my son, I realized that the traditional model with OnPaint event that draws everything that needs to be drawn is not very suitable for what I wanted to have....</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="education" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="turtle" label="turtle" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>As I have been working on the <a href="/education/drawing-trees-with-turtles">Turtle graphics library</a> for my son, I realized that the traditional model with OnPaint event that draws everything that needs to be drawn is not very suitable for what I wanted to have. I wanted to be able to draw when I need to and avoid re-drawing everything that has been drawn so far (or at least do it in an easy way). After a bit of reading and experimenting with <a href="http://en.wikipedia.org/wiki/WxWidgets">wxWidgets</a>, it turned out to be not that difficult. There are four main ideas.</p>]]>
        <![CDATA[<h2>Create a bitmap to be drawn on.</h2>

<pre><code>local bitmap
local mdc = wx.wxMemoryDC()

local function reset ()
  local size = frame:GetClientSize()
  local w,h = size:GetWidth(),size:GetHeight()
  bitmap = wx.wxBitmap(w,h)

  mdc:SetDeviceOrigin(w/2, h/2)
  mdc:SelectObject(bitmap)
  mdc:Clear()
  mdc:SetPen(wx.wxBLACK_PEN)
  mdc:SetFont(wx.wxSWISS_FONT) -- thin TrueType font
  mdc:SelectObject(wx.wxNullBitmap)
end</code></pre>

<p>The code is straightforward: it creates a bitmap using the client size of the frame (the client size excludes border and caption areas); all the drawing is done on that bitmap.</p>

<h2>Draw on the bitmap using <a href="http://docs.wxwidgets.org/2.8/wx_wxmemorydc.html">wxMemoryDC</a>.</h2>

<pre><code>local bitmap
local mdc = wx.wxMemoryDC()

local function line (x1, y1, x2, y2)
  mdc:SelectObject(bitmap)
  mdc:DrawLine(x1, y1, x2, y2)
  mdc:SelectObject(wx.wxNullBitmap)
  if autoUpdate then updt() end
end</code></pre>

<p><code>SelectObject</code> method selects the bitmap and then releases it by selecting the NullBitmap after the drawing operation is completed to allow the object to be cleared when needed.</p>

<h2>Register handlers for <span class="caps">PAINT </span>and <span class="caps">ERASE</span>_BACKGROUND events.</h2>

<pre><code>function OnPaint(event)
  -- must always create a wxPaintDC in a wxEVT_PAINT handler
  local dc = wx.wxPaintDC(frame)
  dc:DrawBitmap(bitmap, 0, 0, true)
  dc:delete() -- ALWAYS delete() any wxDCs created when done
end

frame:Connect(wx.wxEVT_PAINT, OnPaint)
frame:Connect(wx.wxEVT_ERASE_BACKGROUND, function () end) -- do nothing</code></pre>

<p>The <code>OnPaint</code> event handler is very simple: it only draws the bitmap that already has all the content we want to display on the screen. The <code>ERASE_BACKGROUND</code> event handler does nothing as we take care of redrawing everything ourselves. If this is not done, then the user is likely to see flicker that will be created by repeated clearing and redrawing the same frame.</p>

<h2>Handle applications events to avoid "busyness".</h2>

<pre><code>local autoUpdate = true
local function updt (update)
  local curr = autoUpdate
  if update ~= nil then autoUpdate = update end

  frame:Refresh()
  frame:Update()
  wx.wxGetApp():MainLoop()

  return curr
end

local exit = true
frame:Connect(wx.wxEVT_IDLE, 
  function () if exit then wx.wxGetApp():ExitMainLoop() end end)</code></pre>

<p>Combination of <code>Refresh()</code> and <code>Update()</code> calls forces refresh of the content, while calling <code>MainLoop()</code> allows the application to process all pending events (thus avoiding busyness). It seems like <code>wx.wxSafeYield(frame)</code> could potentially be used instead of <code>MainLoop()</code> for the same purpose, but I could not avoid the "hourglass" cursor and the window was slow to respond to mouse events.</p>

<p>Note that the <code>MainLoop()</code> call does not indefinitely block the application as the <code>IDLE</code> event handler immediately exits the main loop when called. As the <code>IDLE</code> event is called only after all the pending events have been processed, this guarantees that the application continues respond to events while drawing the content we need. The "exit" check is only used for the final call of the <code>MainLoop()</code> function when we do want to block until the user closes the application.</p>

<p>The complete code is available <a href="https://github.com/pkulchenko/ZeroBraneEduPack/blob/master/turtle/squared-rays.lua">here</a>. Here is the image it generates:</p>

<p><img src="/images/turtle-squared-rays.png" alt="Squared sun" height="424" width="444" /></p>]]>
    </content>
</entry>

<entry>
    <title>Convert PNG files to an animated GIF</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/convert-png-files-to-an-animated-gif" />
    <id>tag:notebook.kulchenko.com,2012://2.118</id>

    <published>2012-01-19T06:28:22Z</published>
    <updated>2012-01-19T07:08:41Z</updated>

    <summary><![CDATA[I was looking for a way to combine a sequence of PNG files that were generated by the turtle library I have been working on and it turned out to be easy using ImageMagick. The command to use is: &gt;...]]></summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="turtle" label="turtle" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I was looking for a way to combine a sequence of <span class="caps">PNG </span>files that were generated by the <a href="/education/http://notebook.kulchenko.com/education/drawing-trees-with-turtles">turtle library I have been working on</a> and it turned out to be easy using <a href="http://www.imagemagick.org/">ImageMagick</a>. The command to use is:</p>

<pre><code>&gt; convert -delay 40 -loop 1 tree*.png trees.gif</code></pre>

<p>This will generate an animated <span class="caps">GIF </span>file with 40ms delay between images that will play once (<code>-loop 1</code>). You can use <code>-loop 0</code> value to loop indefinitely. Here is the image that was generated:</p>

<p><a href="/images/turtle-trees.gif" onclick="javascript:this.href += '?' + Math.random()"><img src="/images/turtle-trees.gif" /></a></p>

<p>If you do not see any animation and see the grass, it means that the animation is already over (as it loops only once). <del>You can reload the page or click on the image to see it again.</del> It turned out that some browsers (like Firefox) do very aggressive caching of images, including animated images, and will not replay it after page reload and even if you open the same image on a different page (as long as it references the same <span class="caps">URL</span>). I used a simple JavaScript trick to dynamically generate a unique <span class="caps">URL, </span>which will allow you to see the animation when you click on the link:</p>

<pre><code>&lt;a href=&quot;/images/turtle-trees.gif&quot; 
   onclick=&quot;javascript:this.href += '?' + Math.random()&quot;&gt;</code></pre>

<p><code>this</code> refers to the object that the <code>onlick</code> event is triggered for (the a tag in this case) and its <code>href</code> attribute is modified with a random query parameter (which is ignored by the browser in this case).</p>]]>
        
    </content>
</entry>

<entry>
    <title>Drawing trees with turtles</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/education/drawing-trees-with-turtles" />
    <id>tag:notebook.kulchenko.com,2012://2.116</id>

    <published>2012-01-18T23:11:49Z</published>
    <updated>2012-05-19T01:57:59Z</updated>

    <summary>I have been working on a simple Turtle graphics library for my seven year old son and showed him some intermediate results. One of them was a simple fractal tree that looked like this: My son immediately got excited and...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="education" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="turtle" label="turtle" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I have been working on a simple <a href="https://github.com/pkulchenko/ZeroBraneEduPack">Turtle graphics</a> library for my seven year old son and showed him some intermediate results. One of them was a simple fractal tree that looked like this:</p>

<p><img src="/images/turtle-tree-start.png" alt="Turtle tree simple" height="424" width="444" /></p>

<p>My son immediately got excited and wanted to come up with possible ways to change the drawing. His first idea was to color the tree and to make the trunk brown and the leaves green. Then he wanted to make the trunk wider. After that he wanted a smaller tree and grass. Then he wanted to add flowers; at this point I decided to stop, as the tree already looked good and the flowers required a different fractal. Here is the tree we generated:</p>

<p><img src="/images/turtle-tree-color.png" alt="Turtle tree color" height="424" width="444" /></p>]]>
        <![CDATA[<p>The grass pattern is generated using exactly the same logic as the tree, just with a smaller level limit, repeated multiple times with some random shift. The code turned out to be quite simple:</p>

<pre><code>require &quot;turtle&quot;

local f=0.5 -- how much shorter the next tree should be
local a=60 -- what angle to use to turn the branches

function tree(d, n) -- distance to grow, level for recursion
  if n == 0 then return end
  
  local width = pnsz(n) -- set pen size; save current value
  local color = pncr(colr(127,(8-n)*32,0)) -- set pen color

  move(2/3*d)
  turn(a)
  tree(d*f, n-1)
  turn(-a)

  move(1/3*d)
  turn(-a)
  tree(d*f, n-1)
  turn(a)

  turn(7)
  tree(d*f, n-1)
  turn(-7)
  move(-1/3*d) -- note that 1/3+2/3 is not always 1
  move(-2/3*d) -- so move back in the same way as forward

  pncr(color) -- restore pen color to make it work recursively
  pnsz(width) -- restore pen size to make it work recursively

  wait(0.001) -- slow down the drawing
end

turn(-90) -- point turtle up as trees grow up

-- grow a large tree
goto(0, 120)
tree(160, 7)

-- grow a small bush
goto(-80, 120)
tree(80, 6)

-- grow grass
for grass = -180,180 do
  goto(grass+rand(5),120+rand(25))
  tree(8,3)
end

save(&quot;tree&quot;)

wait()</code></pre>

<p>The same method <code>tree()</code> is used to generate both of the trees and the grass; the only difference is the size and the "depth" of the tree: <code>tree(160, 7)</code>, <code>tree(80, 6)</code>, and <code>tree(8,3)</code>.</p>

<p>The <code>turtle</code> library only depends on <code>wxWidgets</code> library for Lua. You can use <a href="https://github.com/pkulchenko/ZeroBraneStudio/downloads">ZeroBrane Studio</a> to run these scrips, the development environment for Lua I developed specifically for students to learn programming and to work on simple script like this one. This development environment already includes <code>wxWidgets</code> library and also allows users to debug applications, which makes it easy to learn how things work. </p>]]>
    </content>
</entry>

<entry>
    <title>The most enthusiastic entrepreneur</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/business/the-most-enthusiastic-entrepreneur" />
    <id>tag:notebook.kulchenko.com,2012://2.121</id>

    <published>2012-01-13T21:42:59Z</published>
    <updated>2012-01-21T21:56:53Z</updated>

    <summary>I participated in the 6th Annual Science and Technology Showcase organized by UW SEBA organization. This was an opportunity to present and discuss a project or technology with potential to start a company. The event included a poster presentation to...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="business" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="startup" label="startup" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I participated in the <a href="http://ugradnews.cs.washington.edu/2011/12/14/6th-annual-science-technology-showcase-sts/">6th Annual Science and Technology Showcase</a> organized by <a href="http://uwseba.com/">UW <span class="caps">SEBA</span></a> organization. This was an opportunity to present and discuss a project or technology with potential to start a company. The event included a poster presentation to 10 judges and a one minute pitch to the group of judges and students (about 150-200 people). My presentation was about the project I have been working on for the past 9 months on <em>developing a robot, software, and educational materials to help middle- and high-school students learn science working on projects with friends</em>. I didn't get the main prize ($1000), but did get a cash prize for <em>the most enthusiastic entrepreneur</em>.</p>]]>
        
    </content>
</entry>

<entry>
    <title>Lua beautifier in 55 lines of Perl</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/programming/lua-beautifier-in-55-lines-of-perl" />
    <id>tag:notebook.kulchenko.com,2011://2.117</id>

    <published>2011-12-19T03:10:37Z</published>
    <updated>2012-05-19T01:58:39Z</updated>

    <summary>I have been working on a Lua IDE (ZeroBrane Studio) and wanted to re-format a large number of Lua files to satisfy my style preferences. I did find a nice Lua beautifier (also written in Lua), but it didn&apos;t handle...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="programming" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="lua" label="lua" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I have been working on a Lua <span class="caps">IDE </span>(<a href="https://github.com/pkulchenko/ZeroBraneStudio/">ZeroBrane Studio</a>) and wanted to re-format a large number of Lua files to satisfy my style preferences. I did find a nice <a href="http://www.userpixel.com/486">Lua beautifier</a> (also written in Lua), but it didn't handle many of the cases that were present in the code I was working with and I didn't want to make any manual changes. I ended up using the same approach, but completely rewrote it in Perl as its regular expressions are a tad more powerful. I'm sure that all this can be re-implemented in Lua, but I didn't have time. Lua regular expressions are simple and quite flexible, but one of the simple features I miss is the ability to add modifiers to groups. For example, I can write <code>%d+</code> to indicate 1 or more digits (same as <code>\d+</code> in Perl), but I can't write <code>(local )?</code> to indicate an optional group. </p>]]>
        <![CDATA[<p>Here is the code I ended up with:</p>

<pre><code>use strict;
use warnings;

use constant INDENT =&gt; '  ';
my($currIndent, $nextIndent, $prevLength) = (0, 0, 0);

while (&lt;&gt;) {
  chomp;
  s/^\s+|\s+$//g; # remote all spaces on both ends
  s/\s+/ /g; # replace all whitespaces inside the string with one space

  my $orig = $_;

  s/(['&quot;])[^\1]*?\1//g; # remove all quoted fragments for proper bracket processing
  s/\s*--.+//; # remove all comments; this ignores long bracket style comments

  # open a level; increase next indentation; don't change current one
  if (/^((local )?function|repeat|while)\b/ &amp;&amp; !/\bend\s*[\),;]*$/
   || /\b(then|do)$/ &amp;&amp; !/^elseif\b/     # only open on 'then' if there is no 'elseif'
   || /^if\b/ &amp;&amp; /\bthen\b/ &amp;&amp; !/\bend$/ # only open on 'if' if there is no 'end' at the end
   || /\bfunction\s*\([^\)]*\)$/) {
    $nextIndent = $currIndent + 1;
  }
  # close the level; change both current and next indentation
  elsif (/^until\b/ 
      || /^end\s*[\),;]*$/
      || /^end\s*\)\s*\.\./ # this is a special case of 'end)..&quot;some string&quot;'
      || /^else(if)?\b/ &amp;&amp; /\bend$/) {
    $nextIndent = $currIndent = $currIndent - 1;
  }
  # keep the level; decrease the current indentation; keep the next one
  elsif (/^else\b/ 
      || /^elseif\b/) {
    ($nextIndent, $currIndent) = ($currIndent, $currIndent-1);
  }

  my $brackets = y/(// - y/)//; # capture unbalanced brackets
  my $curly = y/{// - y/}//; # capture unbalanced curly brackets

  # close (curly) brackets if needed
  $currIndent += $curly if $curly &lt; 0 &amp;&amp; /^\}/; 
  $currIndent += $brackets if $brackets &lt; 0 &amp;&amp; /^\)/; 

  warn &quot;WARNING: negative indentation at line $.: $orig\n&quot; if $currIndent &lt; 0;

  print((length($orig) ? (INDENT x $currIndent) : ''), $orig, &quot;\n&quot;)
    if $prevLength &gt; 0 || length($orig) &gt; 0; # this is to collapse empty lines

  $nextIndent += $brackets + $curly;

  $currIndent = $nextIndent;
  $prevLength = length($orig);
}

warn &quot;WARNING: positive indentation at the end\n&quot; if $nextIndent &gt; 0;</code></pre>

<p>You can update <code>INDENT</code> to match your style; change it to <code>&quot;\t&quot;</code> if you prefer tabs rather than spaces.</p>

<p>The code is not complex, but it handles indentation for opened brackets and curly brackets as well as for anonymous function definitions. For example, it generates this fragment that correctly handles quoted brackets:</p>

<pre><code>  local match = { [string.byte(&quot;&lt;&quot;)] = true,
    [string.byte(&quot;&gt;&quot;)] = true,
    [string.byte(&quot;(&quot;)] = true,
    [string.byte(&quot;)&quot;)] = true,
    [string.byte(&quot;{&quot;)] = true,
    [string.byte(&quot;}&quot;)] = true,
    [string.byte(&quot;[&quot;)] = true,
    [string.byte(&quot;]&quot;)] = true,
  }</code></pre>

<p>and this fragment that includes an anonymous function:</p>

<pre><code>  editor:Connect(wxstc.wxEVT_STC_USERLISTSELECTION,
    function (event)
      local pos = editor:GetCurrentPos()
      local start_pos = editor:WordStartPosition(pos, true)
      editor:SetSelection(start_pos, pos)
      editor:ReplaceSelection(event:GetText())
    end)</code></pre>

<p>The script will also detect and report if something goes wrong and the indentation becomes negative at some point or is greater than zero at the end. Note that the script doesn't re-arrange the lines in any way and only re-formats them, so you can easily check that the number of lines is the same before and after the processing.</p>

<p>It does not handle multi-line comments and strings (of <code>[[ ]]</code> and <code>--[[ ]]</code> style) and any non-matching text inside those can easily throw it off track. It also doesn't detect logical operators that are often used in Lua. For example,</p>

<pre><code>foo = bar and 1
  or 2</code></pre>

<p>will be formatted as:</p>

<pre><code>foo = bar and 1
or 2</code></pre>

<p>This can be fixed by adding brackets:</p>

<pre><code>foo = (bar and 1
  or 2)</code></pre>]]>
    </content>
</entry>

<entry>
    <title>Streaming real-time data from Arduino using AJAX and persistent connections</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/embedded/streaming-real-time-data-from-arduino-using-ajax-and-persistent-connections" />
    <id>tag:notebook.kulchenko.com,2011://2.115</id>

    <published>2011-11-13T00:21:09Z</published>
    <updated>2012-05-19T02:07:45Z</updated>

    <summary>I decided to include an AJAX example that works with streaming data from Arduino in the last version my DHCPLite library. I already showed how the library allows you to point your browser to http://arduino/ and get &quot;Hello, World!&quot; back....</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="embedded" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="ajax" label="ajax" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="arduino" label="arduino" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcplite" label="dhcplite" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I decided to include an <span class="caps">AJAX </span>example that works with streaming data from Arduino in the last version my <a href="/dhcplite/dhcp-server-with-arduino-v0-06"><span class="caps">DHCPL</span>ite library</a>. I already showed <a href="/embedded/dhcp-and-dns-servers-with-arduino">how the library allows</a> you to point your browser to http://arduino/ and get "Hello, World!" back. Now I wanted to actually read the data from a digital or an analog input and show the result as a number and paint it as a circle (using browser canvas).</p>

<p>The model is simple: a request to a <span class="caps">URL </span>like "http://arduino/A1" returns an <span class="caps">HTML </span>page that includes JavaScript code that periodically retrieves data from <span class="caps">URL </span>"http://arduino/A1=", which returns the actual value read from analog pin 1. Similarly, requests for A# and D# retrieve and return current values read from those pins. The result looks like this:</p>

<p><a href="/images/arduino-ajax-1.png"><img src="/images/arduino-ajax-1.png" alt="" height="225" width="368" /></a></p>]]>
        <![CDATA[<p>I have been able to send requests every 40-50ms; I also added 100ms delay between response and the next request to avoid overloading the server.</p>

<p>Before we look at the client-side code, I'll describe the tricks I applied on the server side that some may find useful. </p>

<p>First of all, when I was testing performance on subsequent requests, I noticed that there was occasionally a large gap between receiving a response from Arduino and sending a subsequent request. My original server code was closing a connection after each response and I realized that this was causing additional traffic to close and re-establish the connection. As all modern browsers support <a href="http://en.wikipedia.org/wiki/HTTP_persistent_connection">persistent connections</a> I simply removed <code>RedFly.socketClose(sock)</code> call and added <code>Content-Length</code> <span class="caps">HTTP </span>header to the response to indicate the length of the message (to let the client know when to stop reading the socket). This change had significant effect on performance and the delays between the requests almost disappeared.</p>

<p>There is one thing to keep in mind though: while the persistent connection delivers better performance, the connection is indeed persistent and another client may not be able to connect to the same server on the same port. <strong>[Update 2011/12/20]</strong> To allow multiple connections you may need to open several sockets as in:</p>

<pre><code>h1 = RedFly.socketListen(PROTO_TCP, 80);
h2 = RedFly.socketListen(PROTO_TCP, 80);
h3 = RedFly.socketListen(PROTO_TCP, 80);</code></pre>

<p><strong>[Update 2012/2/25]</strong> It turned out that things are a bit more involved than just opening multiple sockets on the same port. This can only been done when the first socket gets connected (not ahead of time; if you try to run the code above, you will get <span class="caps">INVALID</span>_SOCKET in h2 and h3). Here is the sequence of steps to go through in more detail (as per Andreas Watterott, the <span class="caps">CTO </span>of the company that designed the RedFly shield, who was super helpful in troubleshooting this and other issues):</p>


<ol>
<li>Open the first <span class="caps">LTCP </span>socket in module (for example port no. 8001)</li>
<li>Socket handle returned for this socket would be 1.</li>
<li>Connect this socket to the remote peer socket</li>
<li>You can now open the second socket in module with the same port no. 8001</li>
<li>Socket handle returned for the new socket would be 2</li>
<li>Connect this socket to another remote peer socket</li>
</ol>



<p>The second improvement was in removing multiple <code>RedFly.socketSend...</code> calls. I initially had something that looked like this:</p>

<pre><code>RedFly.socketSend(&quot;HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n&quot;);
RedFly.socketSend(&quot;Content-Length: &quot;);
RedFly.socketSend(...sending content length...);
RedFly.socketSend(&quot;\r\n\r\n&quot;);
RedFly.socketSend(...sending actual content...);</code></pre>

<p>and so on. You get the idea. It turned our that it was a bad way to send the data as there were still noticeable delays between different <span class="caps">HTTP </span>requests. When I was reviewing the sniffed traffic in <a href="http://en.wikipedia.org/wiki/Wireshark">Wireshark</a>, I noticed that there was a large number of duplicate and re-transmitted <span class="caps">TCP </span>packets. I cannot say what was causing them, but, since it was an ad-hoc connection, there was no router involved and both client and the server were sitting close to each other. I switched to building a buffer with the response message and sending it all in one packet, and those retransmissions and the delays they were causing disappeared. </p>

<p>The last thing that tripped me on the server side was related to <code>sscanf</code> function. I wanted to be able to parse <span class="caps">URL</span>s like <code>/A1</code> and <code>/D3</code>, but wanted to have a different handler for <code>/A1=</code> as I wanted the first one to return an <span class="caps">HTML </span>code and the second one to return the actual value. I initially used a simple and straightforward code:</p>

<pre><code>if (sscanf(out, &quot;GET /%1[AD]%2d= HTTP&quot;, &amp;mode, &amp;pin) == 2)....</code></pre>

<p>to check if I indeed get two parameters. But for some reason, this <code>sscanf</code> call returned 2 even when I tested it against <code>&quot;GET /A1 HTTP&quot;</code>. I (mistakenly) assumed that it will return 0 when the equal sign fails to match, but it simply returned the number of matches that happened by that time. I added one more parameter to avoid this issue: <code>if (sscanf(out, &quot;GET /%1[AD]%2d= %1[H]&quot;, &amp;mode, &amp;pin, &amp;ignore) == 3)...</code>. Note that you can use <code>%1[AD]</code> to match 'A' or 'D'.</p>

<p>The client side has all the usual elements: (1) the code is executed from the <code>onload</code> handler (<code>&lt;body onload='doit();'&gt;</code>) to make sure that the <span class="caps">HTML </span>elements are fully loaded as this is needed for references to the canvas element to work, (2) <span class="caps">AJAX </span>calls are implemented using <span class="caps">XMLH</span>ttpRequests (I didn't bother with using alternative methods for old IE versions), and (3) repeated calls are made using <code>setTimeout</code> method to avoid making a new call from the same callback and growing the call stack; <code>getMore()</code> function used in the <code>setTimeout</code> call should be a global one, otherwise <code>setTimeout</code> will not see it.</p>

<p>In most cases you wouldn't need to worry about implementing your own functions to handle <span class="caps">AJAX </span>requests as these functions are provided by all popular JavaScript toolkits, but this wasn't an option in my case. As this <span class="caps">HTML </span>is returned by Arduino and I need to make it work on ad-hoc networks without Internet access, there is no way I can fit jQuery in 32k available to me. In fact, there are only 2k of <span class="caps">SRAM </span>that are available, but there is also program memory that can be used to store string constants, and this is exactly what I used. First, I compressed the code <span class="caps">HTML</span>/CSS/JavaScript code using <a href="http://htmlcompressor.com/compressor.html"><span class="caps">HTMLC</span>ompressor</a> as it was one of the few compressors that could handle <span class="caps">HTML, CSS, </span>and JavaScript in one file and compress it correctly. It reduced the size of the message from 1842 bytes to 1143. Then I split that message into several fragments, as the code would still need to use <span class="caps">RAM </span>to send it and I didn't have 1143 bytes available to send it in one piece. This is something to keep in mind while using <a href="http://www.arduino.cc/en/Reference/PROGMEM"><span class="caps">PROGMEM</span></a> strings: even though you can store large chunks, you may need enough <span class="caps">RAM </span>to work with them.</p>

<p>The last thing of interest is an example of using canvas functions to draw something in the browser:</p>

<pre><code>// do this one time and store generated context
var canvas = document.getElementById('out'); // get canvas element
canvas.width = canvas.height = SIDE;
var ctx = canvas.getContext('2d'); // get canvas context for 2d drawing
// do this every time you need to draw something
ctx.clearRect(0,0,SIDE,SIDE); // clear if there is already something
ctx.beginPath(); 
ctx.arc(SIDE/2,SIDE/2,radius/10,0,Math.PI*2,true); // draw a circle
ctx.fillStyle = '#002D80'; // pick a color to fill the circle
ctx.fill(); // fill the circle</code></pre>

<p>And here is the complete version of the uncompressed code;</p>

<pre><code>&lt;html&gt;
  &lt;head&gt;
    &lt;title&gt;Arduino&lt;/title&gt;
    &lt;script type='text/javascript'&gt;
      var SIDE = 200;
      var DELAY = 100;
      var request = new XMLHttpRequest();
      function getUrl(url, callback) {
        request.onreadystatechange = function() {
          if (request.readyState == 4) {
            callback(request.responseText);
            request.onreadystatechange = function() {};
          }
        }
        request.open('GET', url, true);
        request.send(null);
      };
      function doit() {
        var info = document.getElementById('info');
        var canvas = document.getElementById('out');
        canvas.width = canvas.height = SIDE;
        var ctx = canvas.getContext('2d');
        function circle(radius) {
          ctx.clearRect(0,0,SIDE,SIDE);
          ctx.beginPath();
          ctx.arc(SIDE/2,SIDE/2,radius/10,0,Math.PI*2,true); // shrink radius
          ctx.fillStyle = '#002D80';
          ctx.fill();
          info.innerHTML = radius;
          getMore = function () { getUrl(window.location.pathname + '=', circle); }
          setTimeout('getMore()', DELAY);
        }
        circle(0);
      }
    &lt;/script&gt;
    &lt;style type='text/css'&gt;  
      html, body { width: 100%; height: 100%; }  
      html { overflow: hidden; } 
      body { 
        margin: 0px; 
        font-family: Verdana,Geneva,Georgia,Chicago,Arial,Sans-serif,'MS Sans Serif';
      }  
      #info { 
        position: absolute; padding: 4px; left: 10px; top: 10px; 
        background-color: #FFFFFF; border: 1px solid #002D80; 
        color: #002D80;
        opacity: 0.8;
        -moz-border-radius: 5px; -webkit-border-radius: 5px;
      }
    &lt;/style&gt;
  &lt;/head&gt;
  &lt;body onload='doit();'&gt;
    &lt;canvas id='out'&gt;&lt;/canvas&gt;
    &lt;div id='info'/&gt;
  &lt;/body&gt;
&lt;/html&gt;</code></pre>]]>
    </content>
</entry>

<entry>
    <title>DHCP Server with Arduino v0.06</title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/dhcplite/dhcp-server-with-arduino-v0-06" />
    <id>tag:notebook.kulchenko.com,2011://2.114</id>

    <published>2011-11-13T00:13:36Z</published>
    <updated>2011-11-13T00:20:14Z</updated>

    <summary>This version of the library added options for renewal and rebinding timers and a new example with AJAX processing that streams real-time data from Arduino. The library now works correctly with Mac OS clients, but still doesn&apos;t work with iPhone...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="dhcplite" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="ajax" label="ajax" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="arduino" label="arduino" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcp" label="dhcp" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcplite" label="dhcplite" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dns" label="dns" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>This version of the <a href="https://github.com/pkulchenko/DHCPLite">library</a> added options for renewal and rebinding timers and a new example with <span class="caps">AJAX </span>processing that streams real-time data from Arduino.</p>

<p>The library now works correctly with Mac OS clients, but still doesn't work with iPhone (iOS versions 2 and 4) and iPad (iOS version 5). If someone knows what needs to be fixed, please send me an email.</p>]]>
        
    </content>
</entry>

<entry>
    <title>DHCP and DNS servers with Arduino </title>
    <link rel="alternate" type="text/html" href="http://notebook.kulchenko.com/embedded/dhcp-and-dns-servers-with-arduino" />
    <id>tag:notebook.kulchenko.com,2011://2.112</id>

    <published>2011-10-30T22:45:05Z</published>
    <updated>2012-05-19T02:08:36Z</updated>

    <summary>I have been working on my Arduino-based robot and wanted to have a way to run several devices on the same ad-hoc network and use their host names rather than their IP addresses. It turned out that there was nothing...</summary>
    <author>
        <name>Paul Kulchenko</name>
        
    </author>
    
        <category term="embedded" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="arduino" label="arduino" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcp" label="dhcp" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dhcplite" label="dhcplite" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dns" label="dns" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://notebook.kulchenko.com/">
        <![CDATA[<p>I have been working on my Arduino-based robot and wanted to have a way to run several devices on the same ad-hoc network and use their host names rather than their IP addresses.</p>

<p>It turned out that there was nothing available that worked out of the box. I tried several libraries, but some didn't have <span class="caps">DNS </span>server functionality, some were for the Ethernet library, which wasn't compatible with my shield (<a href="http://www.watterott.net/projects/redfly-shield">RedFly</a>), and some simply didn't work. </p>

<p>First step was to get <span class="caps">DHCP </span>working. The sequence of <span class="caps">DHCP </span>messages is simple, but the format is not so much; it took me several attempts to get the format to be correct enough for the client to take the response. Wireshark was invaluable in showing the messages and giving me the ability to compare correct messages from the <span class="caps">DHCP </span>server in my router with the (initially) incorrect messages from my <span class="caps">DHCP </span>server.</p>

<p>Here is how the sequence of messages looks in Wireshark (this is between Windows Vista as a client and my <span class="caps">DHCP </span>server implementation running on Arduino + RedFly):</p>

<p><a href="/images/dhcp-wireshark-1.png"><img src="/images/dhcp-wireshark-1.png" alt="" height="457" width="616" /></a></p>]]>
        <![CDATA[<p>Note that you can use "bootp" filter to select <span class="caps">DHCP </span>and <span class="caps">BOOTP </span>related messages. The Windows client sends <span class="caps">DHCP</span> Request message, which gets <span class="caps">DHCP NAK </span>in response as the server doesn't have any record of the client's mac address in its table (assuming this is a fresh restart of the server). The client then sends <span class="caps">DHCP</span> Discover message, which gets <span class="caps">DHCP</span> Offer response (the server offered to extend a lease). The client then sends <span class="caps">DHCP</span> Request, which now gets <span class="caps">DHCP ACK </span>as the server already registered the client in its lease table and confirms the lease.</p>

<p>This is the final confirmation message in this session:</p>

<p><a href="/images/dhcp-wireshark-2.png"><img src="/images/dhcp-wireshark-2.png" alt="" height="457" width="616" /></a></p>

<p>At this point the client gets its <span class="caps">DHCP </span>address and can ping the server (in this case Arduino, which has 192.168.2.1 address).</p>

<p>Now the <span class="caps">DNS </span>server comes into play. If you enter "arduino" into the browser the client (or rather its <span class="caps">DNS </span>resolver) will try to resolve this name by sending queries to the registered <span class="caps">DNS </span>servers (or by using other means like mDNS and <span class="caps">LLMNR</span>). This interaction can be seen if you filter captured packets in Wireshark by "dns" or port number "udp.port == 53":</p>

<p><a href="/images/dhcp-wireshark-3.png"><img src="/images/dhcp-wireshark-3.png" alt="" height="483" width="616" /></a></p>

<p>As can be seen in the screenshot, the client sent a query about "arduino.mshome.net" (in this case "mshome.net" is configured as the domain name, which was returned in the <span class="caps">DHCP </span>response as option 15); the server responds with the IP address for this hostname (192.168.2.1). Note that the server responded with "No such name" response to queries about other domains as it had no information and no access to other <span class="caps">DNS </span>servers to forward the request to.</p>

<p>The client then sends an <span class="caps">HTTP </span>request to that address and gets a simple <span class="caps">HTTP </span>response back:</p>

<p><a href="/images/dhcp-wireshark-4.png"><img src="/images/dhcp-wireshark-4.png" alt="" height="172" width="416" /></a></p>

<p>Here is how you'd use this library:</p>

<pre><code>#include &lt;DHCPLite.h&gt;
...
      // for a request that comes to port DHCP_SERVER_PORT (UDP/67)
      buf_len = DHCPreply((RIP_MSG*)buf, buf_len, serverIP, domainName); 
...
      // for a request that comes to port DNS_SERVER_PORT (UDP/53 or TCP/53)
      buf_len = DNSreply((DNS_MSG*)buf, buf_len, serverIP, serverName); </code></pre>

<p>Both <code>DHCPreply</code> and <code>DNSreply</code> methods use the buffer that has the received message to compile a return message; they both return a new message length or 0 to indicate a problem with the received message.</p>

<p>The code for this library is available <a href="https://github.com/pkulchenko/DHCPLite/">here</a>. Give it a try and leave a comment or send me an email if you see a problem.</p>]]>
    </content>
</entry>

</feed>

