8 Jul 2012

Erlang unit testing in Nitrogen: eunit

I like to have some automated testing for my next project, and I was happy to find the unit testing module for Erlang (called eunit) bundled with Nitrogen. If you want to keep your production code and test code separate (good introduction here), then you need to fix your Nitrogen Emakefile.

The default Nitrogen project lives in the nitrogen/site directory.
You'll find the following directories in there:
- ebin
- include
- src
- static
- templates

I've added the directory test to this to host my unit tests
Your Emakefile is also in the nitrogen/site directory.
The default version looks like this:

{
  [
    "./src/*",
    "./src/*/*",
    "./src/*/*/*",
    "./src/*/*/*/*"
  ],
  [
    { i, "./include" },
    { outdir, "./ebin" },
    debug_info
  ]
}.

You should change this into:

{
  [
    "./src/*",
    "./src/*/*",
    "./src/*/*/*",
    "./src/*/*/*/*",
    "./test/*",
    "./test/*/*",

    "./test/*/*/*",
    "./test/*/*/*/*" 
  ],
  [
    { i, "./include" },
    { outdir, "./ebin" },
    debug_info
  ]
}.


After this, everytime you run ./dev compile (shell) or sync:go() (Erlang console) the unit tests are compiled as well. You can run your tests the in the Erlang console with eunit:test(module_name)
   

16 Jun 2012

Start nitrogen at boot on linux

Here's how I start nitrogen on my debian machine. No need to start the erlang vm, the nitrogen app will take care of that.

Step 1
Put this shell script in /etc/init.d/nitrogen.sh

#! /bin/sh
### BEGIN INIT INFO
# Provides:          nitrogen
# Required-Start:    mysql
# Required-Stop:
# Should-Start:     
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start Erlang VM and Nitrogen.
### END INIT INFO

case "$1" in
    start)
        echo "Starting Erlang VM and Nitrogen "
        /nitrogen/bin/nitrogen start
        ;;
    restart)
        echo "Restarting Erlang VM and Nitrogen "
        /nitrogen/bin/nitrogen restart
        ;;
    stop)
        echo "Stopping Erlang VM and Nitrogen"
        /nitrogen/bin/nitrogen stop
        ;;
    *)
        echo "Usage: /etc/init.d/nitrogen.sh {start|restart|stop}"
        exit 1
    ;;
esac

exit 0

Not that I listed mysql as a dependency (must be started first). This is specific for my nitrogen site, but might not be necessary for yours. You can leave it empty if you don't need it.
The sequence number is automagically determined based on the dependencies. 

Step 2
Make sure its owned by root:

  chown root.root /etc/init.d/nitrogen.sh

and executable:

  chmod 755 /etc/init.d/nitrogen.sh

Step 3
Add the proper links in rcX.d directories:

  insserv nitrogen.sh

On older debian versions (before 6) where insserv is not present, do:

    update-rc.d nitrogen.sh start 99 2 3 4 5 . stop 20 0 1 6 .

I  picked 99 as the sequence number here, so I'm reasonable sure any dependencies are started first. The default is 20, which is bit to early if you ask me.

That's it, done.



17 Feb 2012

javascript: too much recursion

Say your website wants to let a user, who happens to be browsing in another tab, know it needs attention. Your only option then seems to be changing the page title to get attention. Facebook does this for new messages in a chat. I thought I'd give it a try.

So I did this:

<script>
function set_title(Msg, T) {
  if (T ==1) {
      document.title("default title");
      setTimeout(set_title(Msg, 1), 1000);
  } else {
     document.title(Msg);
     setTimeout(set_title(Msg, 0), 1000);
  }
}
</script>

But then Firefox said "error: too much recursion"
I tried many different forms of recursion, with and without parameters and globals.

In the end I settled for:

<script>
 document.title(Msg);
setTimeout(document.title("default title"), 1000);
setTimeout(document.title(Msg), 2000);
setTimeout(document.title("default title"), 3000);
setTimeout(document.title(Msg), 4000);
setTimeout(document.title("default title"), 5000);
setTimeout(document.title(Msg), 6000);
</script>

Anyone for a better solution?


12 Feb 2012

Mochiweb, Erlang and Gzip

I run Nitrogen on Mochiweb for two projects,  and I'm very satisfied with this setup sofar.
The only downside is that Mochiweb does not gzip content by default. The (otherwise very reasonable) reason given by its maker (Bob Ippolito) is here.

So I tested how bad the damage was. First with YSlow, a part of the great FireBug tool.
No pages were gzipped indeed, and the best candidates for improvements seemed to be all the javascripts that Nitrogen need on the client:

  <script src='/nitrogen/jquery.js' type='text/javascript' charset='utf-8'></script>
  <script src='/nitrogen/jquery-ui.js' type='text/javascript' charset='utf-8'></script>
  <script src='/nitrogen/livevalidation.js' type='text/javascript' charset='utf-8'></script>
  <script src='/nitrogen/nitrogen.js' type='text/javascript' charset='utf-8'></script>
  <script src='/nitrogen/bert.js' type='text/javascript' charset='utf-8'></script>


Altogether these are 5 download (with 5 times HTTP GET overhead) and about 380K data.
My first action was to minify them and put all in one file. There are lost of tools on the web that do this for you, but do mind that when put in one file they stay in the order specified above. If not, you'll get Javascript error referring to non-existing objects.

I tested it again in YSlow, and I went from 5 files totaling about 380K to one file of about 290K.
Next stop: Pingdom.  The one Javascript file took the longest to load ...

Using wf:header() and and serving a manually gzipped file turn out to be a good solution.
I made a Erlang/Nitrogen module called jscript.erl with this content:

%% -*- mode: nitrogen -*-
-module (jscripts).
-compile(export_all).
-include_lib("nitrogen/include/wf.hrl").

main() ->
    wf:header("Content-Encoding", "gzip"),
     #template { file="./site/templates/miniall.min.js" }.


It adds the gzip enconding to the header and then spits out the minified, gzipped javascript file of only 78K. Works like charm. This is not the best solution, because a browser that does not support gzip (do they still exist?) can't decode it. So it's better to first check if the browser supports gzip (check the header with wf:headers()) and serve the minified javascript file if gzip is not supported.


10 Feb 2012

Erlang, Nitrogen and Mochiweb installation

I absolutely love Erlang and Nitrogen, and I decided to build my next project with.
Both do have quite a steep learning curve, and Nitrogen isn't that well documented yet. But the productivity increase for developing webapplications (compared to for instance php and smarty) is really amazing. I can get so much more done in the same time.

Although nitrogen runs out of the box on my macBook, it does not install smoothly on Linux.
Here's how I got the server for www.akulahjodohmu.com running:

1.
Install the dependencies. I'm running Debian and Erlang is in the repository. But it's an older version so I'll get a more recent one from the Erlang site in step 2. First:
apt-get install libncurses5-dev git openssl libssl-dev

2.
Get the release R14B04 from Erlang.org. Although R15 is available it's still very new and I'm not keen on running a fresh major release on a production system.  R14B03 is been used a lot with Nitrogen, but unfortunately has a ssl Cache bug in it, which will leak memory if you use ssl. This project uses the  Erlang inets and ssl modules to connect to Facebook.

3.
Now tar -zxvf the download and cd into its main directory
run make and after it make install

4.
Next, get the latest Nitrogen source from the website and run tar -zxvf nitrogen.tar.gz
cd into the main directory and run ./apps/get_mochiweb.sh

5.
If you run make now, it will probably complain about this:

{"init terminating in do_boot","Module mochiglobal potentially included by two different applications: mochiweb and nitrogen."}

The solution is to replace the line in ./rel/reltool.config that says:

{app, nitrogen, [{incl_cond, include}]},

with:

{app, nitrogen, [{incl_cond, include},
         {mod, mochiglobal, [{incl_cond, exclude}]},
     {mod, mochijson2, [{incl_cond, exclude}]}]},

This is a small adaption for the newer sources, for older versions look here.

6. now run make rel_mochiweb and you're good to go. Your complete Nitrogen server is in the ./rel/nitrogen directory and can be started from with ./nitrogen start  from the ./rel/nitrogen/bin/ directory. The default site runs on http://[yourhost]:8000/







9 Feb 2012

Limits of Sqlite on iPhone

I've been working on an iPhone app that is supposed to be the implementation of what is to be in my dissertation, and I hit the limitations of Sqlite on IOS. The problem is easy enough to explain. There is a list of preconditions a character/player has to match before a project can be chosen.
The preconditions a key value based, say "assertiveness, 65" (meaning the players value must be higher than 65 for this project).  Now if this is the data:

preconditions                      player charactericstics
Id | key | value                    key | value
1  | A    | 30                         A    | 45
1  | B    | 54                         C    | 42
2  | A   |  65

Then it's clear that project 1 matches.
I first thought to do this by generating a nice SQL query in objective-c and fire it off to Sqlite:


((SELECT * FROM preconditions WHERE key = A and value <= 45) UNION
  (SELECT * FROM preconditions WHERE key <> A EXCEPT 
     SELECT * FROM preconditions WHERE key = A AND value > 45)
) INTERSECT (
 (SELECT * FROM preconditions WHERE key = C  and value <= 42) UNION 
 (SELECT * FROM preconditions WHERE key <> C EXCEPT 
    SELECT * FROM table WHERE key = C AND value > 42))

It's very nice to use the set operations UNION and INTERSECT, because all duplicates are removed as wel. But guess what? Sqlite does not support parentheses around the operands of the set operator. You're only allowed to use them around subqueries. So much for this query, precendence is important and I need those parentheses.

My next choice was to load the preconditions in memory (player characteristics are already in memory), loop them, and in each loop purge the projectid that is violating a current constraint. In objective-c:

// loop over player characteristics
for (NSString *keyc in characteristics) {
        NSNumber *valc = [characteristics objectForKey:keyc];
        //loop over preconditions
        for (IdKeyValue *p in preconditions){
            if ([keyc isEqualToString: p.key] && ([valc intValue] < p.value)) {
                //remove all p.oid from preconditions
                for (IdKeyValue *t in preconditions){
                    if (t.oid == p.oid) [preconditions removeObject:t];
                }   
            }   
        }   
    }

The problem is that in the most inner loop, I'm modifying the most outer loop. It turns out that is not a particularly good idea on IOS:

Terminating app due to uncaught exception 'NSGenericException', reason: '*** Collection <__NSArrayM: 0x4b80700> was mutated while being enumerated.(

The easy, but not elegant, solution would be to just list all possible keys when creating the preconditions, and for each project insert value "0" for those key's not mentioned for that project.
The data would look like this:

preconditions                      player charactericstics
Id | key | value                    key | value
1  | A    | 30                         A    | 45
1  | B    | 54                         C    | 42
1  | C    | 0
2  | A    | 65
2  | B    | 0
2  | C    | 0

And the SQL query would be:

SELECT * FROM preconditions WHERE (key = A AND value <=45) INTERSECT 
SELECT * FROM preconditions WHERE (key = C AND value <=42) 
 
I think this should work, but it doesn't feel good.