Skip navigation

Nicolas Cannasse

warning: Creating default object from empty value in /var/www/vhosts/ on line 33.

This years FOTB was special. At the end of my session I showed a sneak preview of project Hiddenwood. I demonstrated complete playback of Audiotool tracks on stage — in a browser. Now that does not sound too special…

But then again, the playback was done using JavaScript only and calculated in realtime.

Audiotool is a complex piece of software so you might ask how one could torture themselves by implementing it in JavaScript? We didn’t. Instead we started building our own vision of a cross-platform application framework a couple of months ago.

Introducing project Hiddenwood.

Hiddenwood is a collection of libraries and tools specifically designed to support different devices and platforms. The core libraries are the “driver layer” and always platform-specific with a platform-independent interface.
On top of that we provide a basic layer of libraries like our UI system, animation framework or managed collections which guarantee 0% garbage collection activity and have been battle-tested in Audiotool.

The framework is all about speed and consistency. The rendering pipeline is optimized for OpenGL and although we offer something similar to Flash’s display list a lot of features are not available because they would compromise the speed.

Speaking about speed: we are always interested in staying as native as possible on our target platform. So for the browser we emit JavaScript, for Android you will get the full DalvikVM performance and for the desktop you will get JVM performance. This approach has also another very important aspect. If you want to go platform-specific for certain features you can do that.
For instance if we want to render Audiotool songs on the server using a fork-join pool for our audio calculation this is possible and might not make sense on an Android device.

You write Java code and the supported platforms are native desktop applications, Android (minimum requirements are Gingerbread and OpenGL ES 2.0) and modern browsers. Now for browsers we even go one step further and support multiple options. That means if WebGL is not available we simply fallback to a normal canvas based render-engine. The same applies to some of the Android drivers.

iOS is of course important as well and we are actively researching the best option that will give us the most flexibility and performance.

We are currently working on two real applications built with Hiddenwood. So far it is a real pleasure to enjoy quick build times and simply test what you want on the desktop with great debugging capabilities. When you are ready you can try the same app on Android or in the browser — which might take a little bit longer to compile.

Because we see Hiddenwood as an application framework there are a lot of goodies built-in like a sprite-sheet based class generator. Think Image mixerBackground = Textures.mixer.background(); where mixer was the folder name and background the name of the file.

We believe that as a developer you really do not care about what kind of technology you are using and just want a great result. We also think that you should be able to reuse platform-independent code across multiple projects. However we do not want to take power away from the developer because if you know what you are doing: go for it.

Of course we are not the only ones with this idea. Nicolas Cannasse saw the signs years ago and invented haXe which gives you a comparable experience and Google released playN a couple of weeks ago which takes a similar approach (and requires another 25 committers :P).

But when we started Hiddenwood we wanted the Java tooling experience and playN was not public at that time. We also think that a game engine is not what you want to use for all kinds of applications. So we like to be able to give people the freedom to build their own game engine on top of Hiddenwood — and calculate physics in a different thread peut-être.
Speaking about threading: the only possible solution that works across all platforms is a shared-nothing architecture which we put in place. However if you write platform specific code you can use of course everything the platform offers and a lot of the Hiddenwood core libraries like the network- or cache-layer make use of multiple threads.

In the end what makes Hiddenwood special in my opinion is that we do not believe in write once run anywhere because that just does not make sense. The essence and philosophy behind Hiddenwood is to write platform-agnostic code using kickass-libraries and being able to reuse that. Audiotool on a tablet would look completely different from Audiotool running in a browser. And Audiotool on iOS would probably be also a little bit different from Audiotool on an Android device because there are simply different paradigms you should respect.

I hope that we can share more information with you soon. With the news of mobile Flash Player being deprecated and the ongoing demand for cross-platform development we have exciting times ahead of us. I am also super excited about the (beautiful <3) applications which we are going to release in the not so distant future.

flattr this!

Your rating: None

Finally I found a bit of time this weekend to do some other tests with expressions evaluation.

The results are pretty interesting, even if obvious from some point of view. I took the ExpressionEvaluator I wrote as example for the first post about this topic, and then I edited a bit the code adding just in time AS Bytecode compilation. Thanks to that, expressions evaluation is much more fast and always safe because it runs in its own ApplicationDomain.

You can download the sources here, that includes the edited code and a test file. The test file runs 1 milion of iterations and may hang your browser or at worst case your system. Reduce the value of the ITERATIONS constant if you are not sure about the power of you machine.

If you want to read a bit more details about that, click on the link below to continue reading.

I did some simple modification to the existing classes:

  • Added a compile method to the IExpression interface;
  • Added the symbolNames getter to the SymbolTable class, to retrieve a list of all the symbols defined inside the symbol table.
  • Converted hxasm to Actionscript (finxing some small issues to avoid to include all the Haxe support classes);
  • Implemented the compile method on all the AST nodes using HxASM;
  • Added an SWFCompile class that takes an IExpression and a SymbolTable and produces an SWF;
  • Edited the CompiledExpression class adding a compile method that generates an SWF starting from an IExpression.

Adding just in time compilation was easy a really quick thanks to HxASM. HxASM is an Haxe library written by Haxe's author (Nicolas Cannasse) that makes you able to write low level bytecode instructions in ActionScript an generate a ByteArray that contains an (hopefully) valid SWF. The SWF can then be loaded by Loader.loadBytes and executed.
It have been easy to port HxASM to ActionScript too, thanks again to the Haxe compiler which is able to generate AS3 code starting from Haxe code.

Compilation was achieved easilly because the expressions we are compiling are really simple. The compiler generates a class called CompiledExpression that has a method execute that can be called to execute the expression. All the values inside the symbol table are exposed as public properties.

Just to make you understand better, givin this expression:

sin( x * 10 + y - 11 )

SWFCompiler will generate the bytecode for this class:

class CompiledExpression
  public var sin: *;
  public var x: *;
  public var y: *;
  public function execute(): Number
    return sin( x * 10 + y - 11 );

As you can see the type of the class variables is generic, but you can improve the generator to understand if a variable should contain a number or a function.

The generated SWF can be loaded and run easilly:

var loader: Loader = new Loader();
var parser: Parser = new Parser(
  new Scanner( "sin( x / ( 4 / 2 * 2 - 2 + 2 * x / x ) ) * 100" ) );

var compiled: CompiledExpression = parser.parse();
var data: ByteArray = compiled.compile();

loader.contentLoaderInfo.addEventListener( Event.COMPLETE,
  function( event: Event ): void
    var info: LoaderInfo = ( as LoaderInfo );
    var klass: Class = (
      info.applicationDomain.getDefinition( "CompiledExpression" ) as Class );

    var cp: Object = new klass();

    cp.x = 10;
    cp.sin = Math.sin;

    trace( "Result", cp.execute() );

  } );

loader.loadBytes( data, new LoaderContext( false,
  new ApplicationDomain( ApplicationDomain.currentDomain ) ) );

I build a simple test (shipped with the sources) that runs 1 milion times an expression using 3 methods: Evaluation, Compiled and Native.

Results (on my Mac Pro 8 Core, 4GB ram) are obvious:

eval 1000000 iterations: 12939
compiled 1000000 iterations: 848
native 1000000 iterations: 371

As you can see native execution is faster, but compiled version runs really well (you might want to optimize the generated code to make it run faster) and it is many times faster then AST evaluation.

If you want to run a complex expression many times and dont want to stress your Flash Plugin, you can use the compiled version.

Gotta run, see u next time!

Your rating: None