JOSHUA HOLBROOK github : twitter

Interactive Plotting with Node.js (posted 26 Feb 2011)


A near-indespensable part of any scientific programming environment is an interactive plotting tool. For example, MATLAB and Octave have plot() and friends, python has matplotlib, R has its share of plotting tools, and such things as excel and vtk are often used alongside environments that don’t lend themselves very well to the whole “exploratory programming” thing.

Most of the time, plotting in javascript is used for real-time dashboard type stuff. For example, a lot of people throw smoothie charts up and plot data streams for real-time monitoring of things like CPU usage and temperature and what-not. Pretty awesome, but not quite what I wanted. So, I took a few hours, learned me some new tools, and built what I’m calling “Plolt.”

Plotting library: Flotr

For the plotting library, after some research I chose flotr. There are a metric ass-ton of plotting libraries out there, but flotr was one that met my needs:

  1. Plots to either SVG or canvas. Flot fails this one because it uses standard DOM objects to make its plots. This means, for example, that a y-label is out of the question.

  2. Has decent documentation. This is where g.raphael fails. This is probably because the author hasn’t really had the time to polish up the g.raphael project. This is a shame, because I love raphael.

  3. Has good support for scatter plots. Grafico fails here, because it primarily supports line plots. This is a shame, because its plots look dead sexy, and I love raphael.

  4. Has a sane API for simple scatter plots. CanvasXPress makes really pretty plots, but its support for simple 2-d and 3-d data is lacking, as it was designed to work primarily with annotated data. So, it fails. That said, its 3D plot may be the best solution out there for 3-d that I’ve seen.

  5. Doesn’t try to make things look TOO flashy. If the authors mention Edward Tufte, that’s usually a good sign. As an aside, I think readers of Tufte might cargo cult it up a bit, though.

Serving Up Plots with Express and Jade

I decided to give Express a shot. I’ve been meaning to for a while! It turns out that Express is pretty easy. The Express part of the code looks, basically, like this:

var app = express.createServer();

app.use(express.staticProvider(__dirname) );
app.set('views', __dirname);

app.get('/', function(req, res) {
    res.render('index.jade', {layout: false});
});

You can see that it’s PRETTY STRAIGHTFORWARD in terms of javascript. All I did was set up a few options and told it to serve up html generated with jade, from index.jade, when people visited the site’s root. Awesome!

It turns out Jade isn’t too bad for HTML, either. It has its warts like everything else (for example, 2-space indents instead of 4-space ;) ), but all in all it’s helpful. The Jade code looks like this:

!!! 5
html(lang="en")
  head
    title="roflol"
    script(type="text/javascript",src="js/prototype.js")
    script(type="text/javascript",src="js/flotr.js")
    script(type="text/javascript",src="dnode.js")
    script(type="text/javascript",src="client.js")
  body
    h1 THIS IS THE REMIX
    #remix(style="width:640px;height:480px;")
      p RIGHT HERE BUDDY

I wasn’t very professional when I made this, I’ll admit. I wish there was a cleaner way to do script stuff in Jade, but I’m not complaining too much. Anyways: All this really does is add a title (“THIS IS THE REMIX”) and a div called “remix” with a paragraph in it with some filler text.

Using DNode for Page/Server Communication:

This part was ripped pretty much directly from SubStack’s wonderful article on Ad-Hoc Pubsubs with DNode. I have to admit that the x.bind() magic still trips me out a bit. But, conceptually, all we did was make the web page subscribe for updates on data, in the form of arguments to Flotr.draw():

//This is in client.js
document.observe('dom:loaded', function() {
    DNode.connect(function(server) {
        var emitter = new EventEmitter;

        emitter.on('data', function(xy,opts) {
            Flotr.draw($('remix'), xy, opts);
        });

        server.subscribe(function() {
            emitter.emit.apply(emitter, arguments);
        });
    });
});

If you’ve read SubStack’s article, this should look very familiar, save for the Flotr.draw() line.

Plotting things in flotr is done like this:

Flotr.draw( prototype_selector, data, options)

so what we do is basically set the Prototype selector and pass on the data and options. In fact, it could probably be written as

emitter.on('data', Flotr.draw.bind(null, $('remix')))

but I’m not so hot with bind’s use yet.

Using DNode for Server/REPL Communication:

DNode was used a second time to make RPCs from the REPL to the Express app. This is in a module called “plolt.js” :

//plolt.js
var DNode = require('dnode');

module.exports = function(xy, opts) {
    DNode.connect(6061, function(server) {
        server.plot(xy, opts);
    });
}

and this is in the server that hosts the Express app:

//server.js
DNode(function() {
    this.plot = function(xy, args) {
        publish('data', xy, args);
    }
}).listen(6061);

Basically, sending arguments to the plolt.js module uses DNode to communicate with the running server. Interestingly enough, this mirrors the approach that ipython uses—that is, ipython runs what’s necessary to render matplotlib plots in another thread, and uses ipc’s to allow one to generate a plot in ipython and continue going. This is why one must call ipython –pylab in order to use matplotlib without a bunch of heartache. Or, at least, this is my understanding.

The Exciting Part: Use!

Here’s a screenshot showing everything in all its glory:

The top window shows the server running. The bottom window shows the REPL, and the call to plot(). The browser window shows whatever you last told it to plot via the plot() call.

What next?

This is obviously just a proof of concept. What would make this MORE BADASS? Here are some of my ideas:

  • Exposure of more of the plotting API.

  • A prettier web page, with more meaningful text.

  • image-based output. If one were using a svg-based lib, you would already have them in vector-based format, which is technically superior for plots.

  • The ability to have multiple figures at once. In fact, you could make all figures (semi-permanent) and have the call to plot() return a url with a key identifier (a la imgur).

I’m sure there are others. I’m sure I’ll be really slow to implement any of them, if I ever do. But you know it’d be badass!

Try it yourself!

Get the source code here: https://github.com/jesusabdullah/plolt

In order to run the server, you will need DNode, Express, Jade and underscore, and all the dependencies those require. If you have npm, it should be easy to install all of them. In fact, you should be able to use npm with the included package.json, though it’s untested.

Happy plotting!