How to use native libraries on Node.js with Emscripten

Emscripten is an LLVM-bitcode to Javascript compiler. It Compiles C/C++ code into highly optimized ASM.js Javascript or webassembly which allows you to run code at near native speed without requiring you to use complex FFI bindings.

Setting up Emscripten

Pulling down the emscripten compiler is rather straightforward:

1. You have the option of downloading the latest release linked in the documentation.
2. Or cloning the project off of the github repository (recommended if you plan on compiling to webassembly).

Once you have the project, cd into the directory and have Emscripten install the latest toolchain by issuing the following commands

# Fetch the latest registry of available tools.
./emsdk update

# Download and install the latest SDK tools.
./emsdk install latest

# Make the "latest" SDK "active" for the current user. (writes ~/.emscripten file)
./emsdk activate latest

This comes with a release of the llvm compiler toolchain which we would have to setup as our clang and clang++ compilers.

Fortunately, there’s a handy shell script in the project root that we can use to do just this. It also adds tools from the emsdk to our path for easy access.

# Activate PATH and other environment variables in the current terminal
source ./emsdk_env.sh

Now you should be able to verify that emcc, the emscripten C compiler is available by running the following command.

$ emcc -v

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.37.28
clang version 4.0.0  (emscripten 1.37.28 : 1.37.28)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/vagrant/emsdk/clang/e1.37.28_64bit
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0

...

Compiling native code

At this point we are ready to hack around with compiling a C/C++ program. Let’s attempt compiling a file we can call fizzbuzz.cpp with content,

#include <iostream>

using namespace std;

int main()
{
	for (auto i = 1; i <= 100; ++i)
	{
    	if (i % 3 == 0 && i % 5 == 0)
        	cout << "FizzBuzz";
    	else if (i % 3 == 0)
        	cout << "Fizz";
    	else if (i % 5 == 0)
        	cout << "Buzz";
    	else
        	cout << i;
    	cout << endl;
	}
}

This is the classic FizzBuzz program implemented in C++ 11 (note the use of the auto keyword). We can compile it to Javascript using emscripten by running emcc

emcc fizzbuzz.cpp -std=c++11 -o fizzbuzz.out.js

This will output a file fizzbuzz.out.js which we can run using node. The resulting Javascript is pretty lengthy and code dense primarily because it contains things like comments, runtime support code with features like bounds checking, as well as a good chunk of libc. Using optimization flags would help in removing comments and reducing the size of the resulting output.

$ node fizzbuzz.out.js

1
2
Fizz
4
Buzz
/* Truncated */

Passing html as the output file extension tells Emscripten to generate an HTML page template bootstrapped to run your code

emcc fizzbuzz.cpp -std=c++11 -o fizzbuzz.html

To compile down to webassembly, all you need to do is add the WASM compiler flag

emcc fizzbuzz.cpp -std=c++11 -s WASM=1 -o fizzbuzz.html

This will generate a fizzbuzz.wasm module along with fizzbuzz.js to load the wasm module.

Compiling native libraries

While compiling your own native code is pretty cool, Emscripten’s true value comes in including native libraries in your project.

To do this, your libraries have to be compiled into LLVM bitcode.

Compiling them into standard static libraries won’t work. The best candidates for compilation via emscripten are standalone portable libraries. This is because often the result of the build is intended to target the browser. Complex builds with OS specific dependencies will require additional work to port over.

To walk through the process we’ll be building zip; A lightweight portable zip library based on the miniz compression library.

We can begin by cloning the repository and navigating to the directory.

$ git clone https://github.com/kuba--/zip.git
$ cd zip

Next we’ll build the library using emscripten wrappers around cmake and make. These replace the default system C/C++ compilers with emscripten during the build process.

mkdir build && cd build
emcmake cmake .. # Use CMake to generate Makefile
emmake make # build the project

At this point we should have an static archive libzip.a that contains the llvm bitcode we need to link against when building our final executable.

Let’s write some code that will be using this library.

#include <stdio.h>
// Header for zip
#include <zip.h>
// Emscripten header
#include <emscripten.h>

#define FS_PREFIX "fs"

// Prepend file system path
const char* buildRelativeFilePath(const char* filename) {
	char *path;
	asprintf(&path, "%s/%s", FS_PREFIX, filename);
	return path;
}

int main()
{
	// mount the current folder as a NODEFS instance
	// inside of emscripten
	EM_ASM({
    		var directory = '/' + UTF8ToString($0);
    		FS.mkdir(directory);
    		FS.mount(NODEFS, {root : '.'}, directory);
	}, FS_PREFIX);

	// construct zip file
	struct zip_t *zip = zip_open(buildRelativeFilePath("foo.zip"),
    		ZIP_DEFAULT_COMPRESSION_LEVEL, 'w');
	{
    	zip_entry_open(zip, "foo-1.txt");
    	{
        		const char *buf = "WillowTree rocks!";
        		zip_entry_write(zip, buf, strlen(buf));
    	}
    	zip_entry_close(zip);
	}
	zip_close(zip);
}

Here we’re including a header file from zip so we can use functions built into the project. We’re also including the emscripten header so we can mount the node file system instance. This will allow us to write outside of the emscripten virtual file system.

#define FS_PREFIX "fs"

EM_ASM({
	var directory = '/' + UTF8ToString($0);
	FS.mkdir(directory);
	FS.mount(NODEFS, {root : '.'}, directory);
}, FS_PREFIX);

This mounts the vitual file system path "/fs" to our current working directory. So to write a file foo.txt, we’d need to write it to fs/foo.txt.

Finally, we can now compile our source using em++, the emscripten counterpart to g++ or clang++. Here we statically link to the archive as well as pass it’s path and the zip project’s source folder to include the header files.

em++ main.cpp -lzip -L<path to libzip.a> -I<path to zip>/src -o zipit.out

This gives us a file zipit.out.js we can execute which creates a zip file containing a single text file as expected.

Using Emscripten Ports

Building libraries yourself can be quite tedious especially when faced with large projects or situations where a library contains dependencies on other shared libraries. A great example of this would be the popular libpng or SDL which require some work to make compatible with Emscripten. A great initiative underway is the Emscripten ports project that aims to make it easy to add such projects to your builds. For example, adding libpng to your project is as easy as running

emcc make_png.c -s USE_LIBPNG=1

You can get a full list of Emscripten ports available by running

$emcc --show-ports

Available ports:
	zlib (USE_ZLIB=1; zlib license)
	libpng (USE_LIBPNG=1; zlib license)
	SDL2 (USE_SDL=2; zlib license)
	SDL2_image (USE_SDL_IMAGE=2; zlib license)
	ogg (USE_OGG=1; zlib license)
	vorbis (USE_VORBIS=1; zlib license)
	bullet (USE_BULLET=1; zlib license)
	freetype (USE_FREETYPE=1; freetype license)
	SDL2_ttf (USE_SDL_TTF=2; zlib license)
	SDL2_net (zlib license)
	Binaryen (Apache 2.0 license)
	cocos2d

Conclusion

In this post, we’ve seen Emscripten in action as a tool to make compiling native code to Javascript and Webassembly making native code even more portable. Supporting multiple platforms on projects that require extremely complex business logic would only require you to write it once in native code and recompile it for all your respective targets with the option to support all platforms including Android, iOS, Desktop and Web. There are already quite a few projects taking advantage of it today and the rapid development of Webassembly makes the future look even more exciting.