New lexer and parser in IronJS
I’ve been thinking about replacing the lexer and parser IronJS has been using for a year now, which is an ANTLR generated LL(*) parser. The main drive behind this is that I’ve been wanting to shed the two DLL dependencies the parser caused, first the runtime for ANTLR (Antlr3.Runtime.dll) and then the parser itself (Xebic.ES3.dll) – since it was C# it wasn’t possible to integrate the code into the IronJS F# code base and it had to be linked as a separate assembly.
I’m glad to announce that I’ve finally gotten around to do this and that the new F# based lexer and parser were pushed to the master branch on github earlier today. I also decided to remove the dependency on Microsoft.Dynamic.dll which I only did about ten calls into. This means IronJS now only requires FSKit other then itself, the plan is to merge the FSKit functionality that IronJS requires into the IronJS project itself so it will only be one DLL.
Another great benefit of rewriting the parser in F# is a pretty nice speed boost, if I can direct your attention to the chart below you will see that the new lexer and parser is about eight times faster on the jQuery 1.5.1 (uncompressed) source code. This of course means that IronJS i getting even faster then it was.
Also, keep your eyes open for the first 0.2 beta that will arrive shortly.
I got a question on IRC on how the profiling was done, so here’s a description of it.
- System.Threading.Thread.CurrentThread.Priority was set to System.Threading.ThreadPriority.Highest
- Timing was done with the System.Diagnostics.Stopwatch class
- The source code was loaded into memory before lexing and parsing so no disk penalty would occur
- The machine, which is a i7 Quad Core with 8Gb of ram, was restarted between each test and as many processes as possible were killed when Windows was done booting
- The projects were compiled in release mode with full optimizations and no debug info
- Each test was ran ten times before timing started to make sure all assemblies were loaded in memory and there would be no JIT overhead
- After the ten warm-up runs the test was ran 100 times, the ten fastest were picked and averaged
If there are any flaws in the above process please do point them out and I will re-do the test and post new results.