Wiring up dependencies in node.js – wrap up of my best practices

Quite a lot of friend developers that make their first baby steps with node.js keep asking me about the “right” way to share database resources respectively dependencies in general among controllers, business logic or service objects. Now, there’s actually one universal wisdom in the world of node.js: there is no “right” way of doing things (have a look at the controller samples from express.js ).

There are two good reasons why: First, most node.js frameworks including its core are very basic, offering modular and highly simplistic, fundamental but reusable code. They don’t make any assumptions about how you will use them, leaving it up to you to wire everything together as you want. Second, Javascript as a functional language offers many more options to shoot yourself in your foot than class oriented / C-like languages. What’s missing (and maybe even unwanted) in the node.js universe is a sophisticated framework like Spring, Rails or Symfony that offers best practices under the hood you could rely on.

In this article I’d like to tell my story of how I first tried to transfer my knowledge from these “sophisticated” frameworks (usually IoC containers) to express.js just to recognize the far simpler, nodejs-“native” approach to achieve the same results. The first three examples I’m presenting are just evolutions of ideas I had until I noticed that in node.js you can take much simpler ways – so be warned that some of the upcoming code might seem unnecessarily blown up: the resolution you might want to comment on can be found at the very end.

The old world

Connecting to some kind of server-side database layer is a pretty straight forward task in your favorite non-JS language. Have a look at some pseudo-code:

Connection con = Driver.connect("some://driver.specific@connection.string:for/your/database");
ResultSet result = con.query("SELECT foo FROM bar WHERE id=1337");
Entities entities = new ArrayList;
foreach(result as r) { 
    Entity e = Driver.hydrate(r); 
    entities.add(e);
}   

If you’re a real node.js greenhorn, let me quickly explain why this kind of code will never work on node. Since Javascript (really!) executes in one thread at a time lengthy operations like connecting or querying a database must not block the main execution loop – otherwise the application won’t be able to respond other incoming requests. Instead of waiting for the database to return a connection (like e.g. Java does), V8 will immediately execute the subsequent code. Regarding the example above con might not have been initialized yet when `con.query is executed. In Javascript you handle this kind of asynchronous events using callbacks, so in node.js the above example could be pseudo-coded as:

var entities = [];

driver.connect(connectionString, function(err, con) {
    var hydrator = new driver.Hydrator;
    con.query("SELECT ...", function(err, result) {
        _.each (result, function(r) {
            hydrator.hydrate(r, function(err, entity) {
                entities.push(entity);

                //business logic here

            });
        });
    });
});

You immediately notice the main problem in functional code: some call it the “Javascript pyramid of death”. It’s built on subsequently registered callbacks. I won’t dig deep into solutions to that issue here (promises are currently the best solution and they’re widely adapted ) but I want you to have a look at the first line that connects to the database and serves as the root of our pyramid. On platforms like PHP or Java you would have a single place where you connect to your database. Then you would either wire that connection to clients that want to use it or ask some container to hand it over back to you (and initialize it if it wasn’t before). Let’s have a look at that pattern in a container managed environment (far from being exactly IoC, but you should get the idea):

“Java” pseudo-code:

class Container {
    resources = array();  
    con = null;

    public function getConnection() {
        if (null == this.con) {
            this.con = Driver.connect(...);
        }
        return this.con;
    }
}  

class FooController {

    @Inject("container")
    private container;

    db = this.container.getConnection().query("...");
}

The idea behind this pattern is: there’s some godlike mega-registry (the IoC-Container) knowing, configuring and instantiating all your dependencies. If you need something you either annotate your dependencies and let Mr Registry inject it at startup time or call Mr Registry and ask for a fully configured and initialized resource (service, bean, you name it).

IoC-like coding in express.js / node.js

Adapting this pattern in node.js leads to rather uncomfortable code. Lets start with an app.js to illustrate that (Again: beware that I wouldn’t recommend to use this kind code but you could definitely do so).

/code

app.js

var express = require("express");
var http = require('http');
var Controller = require('./controller.js');
var Container = require("./ioc.js");

var app = express();
app.use(express.bodyParser());

app.set('port', process.env.PORT || 3000);

var container = new Container();
new Controller(container).route(app);

app.use(app.router);

http.createServer(app).listen(app.get('port'), function(){
    console.log('Express listens on port ' + app.get('port'));
});

ioc.js

var sqlite3 = require("sqlite3");
var Container = module.exports = function() {
    var db = null;
}

Container.prototype = {
    initializeDb: function(db) {
        db.run("CREATE TABLE testing " +
            "(id INTEGER PRIMARY KEY AUTOINCREMENT, " +
             "info TEXT)", function(err) {
            console.dir(err);
        });
    },
    getDb: function() {
        if (null == this.db) {
            this.db = new sqlite3.Database(":memory:");
            this.initializeDb(this.db);
        }
        return this.db;
    }
}

controller.js

var Controller = module.exports = function(container) {
    this.container = container;
}

Controller.prototype =  {
    indexAction: function(req, res) {
        var db = this.container.getDb();
        db.all(
           "SELECT * FROM testing",
            function(err, rows) {
                res.json(rows);
            });
    },
    fooAction: function(req, res) {
        var db = this.container.getDb();
        db.get(
            "SELECT * FROM testing WHERE id = $id",
            {$id:req.params.id},
            function(err, row) {
                res.json(row);
            });
    },
    addAction: function(req, res) {
        var db = this.container.getDb();
        db.run(
            "INSERT INTO testing (info) VALUES ($info)",
            {$info: req.body.info},
            function(err) { //this: statement
                res.json({ lastId: this.lastId});
            });
    },
    route: function(app) {
        app.get('/foo', this.indexAction.bind(this));
        app.get('/foo/:id', this.fooAction.bind(this));
        app.post('/foo', this.addAction.bind(this));
    }
}

If you run this example and access GET /foo it’ll be called back with the error message “Error: SQLITE_ERROR: no such table: testing“. Notice that initializeDb has been called by getDb but while it tried to execute the initial CREATE statement, the single thread already went on and executed the index action. Since the initializeDb callback has not been handled yet (it’s going to be handled right after the index action has finished) the SELECT statement cannot find the table yet.

To remedy that situation, we could use callbacks in our client code, like this:

fooAction: function(req, res) {
    this.container.getDb( function(db) {
        db.get("SELECT ...", {}, function() { ... });
    });

That way we’re littering the container’s getDb interface with a callback parameter. Instead, people choose to use the so called promise pattern at this point. There are some libraries out there that get the job done; Q is one of the most powerful of them and besides many other features it offers a deferred interface that deals with that problem.

IoC-like code with promises

/code

container.js

Container.prototype = {

    initializeDb: function(db, callback) {
        db.run("CREATE TABLE testing " +
            "(id INTEGER PRIMARY KEY AUTOINCREMENT, " +
            "info TEXT)", callback);
    },
    getDb: function( ) {
        var deferred = Q.defer();
        if (null == this.db) {
            var self = this;
            this.db = new sqlite3.Database(":memory:");
            this.initializeDb(this.db, function(err) {
                if (err) {
                    deferred.reject(new Error(err));
                } else {
                    deferred.resolve(self.db);
                }
            });
        } else {
            deferred.resolve(this.db);
        }
        return deferred.promise;

    }
}

controller.js

    ...
    indexAction: function(req, res) {
        this.container.getDb().then( function(db) {
            db.all("SELECT * FROM testing",
                function(err, rows) {
                    res.json(rows);
                });
        });
    }
    ...

Starting the app after container setup has finished

That looks only little better but definitely not really usable yet. So lets take a last attempt to fix things up. Let’s tell the container to initialize everything and start the application once the basic initialization has finished.

/code

app.js

var container = new Container();
container.initialize().then( function() {
    new Controller(container).route(app);
    app.use(app.router);

    http.createServer(app).listen(app.get('port'), function(){
        console.log('Express listens on port ' + app.get('port'));
    });
});

container.js

Container.prototype = {

    initializeDb: function(db, callback) {
        db.run("CREATE TABLE testing " +
            "(id INTEGER PRIMARY KEY AUTOINCREMENT, " +
            "info TEXT)", callback);
    },
    getDb: function( ) {

        if (null == this.db) {
            var deferred = Q.defer();
            var self = this;
            this.db = new sqlite3.Database(":memory:");
            this.initializeDb(this.db, function(err) {
                if (err) {
                    deferred.reject(new Error(err));
                } else {
                    deferred.resolve(self.db);
                }
            });
            return deferred.promise;
        } else {
            return this.db;
        }

    },
    initialize: function() {
        var dfd = Q.defer();
        this.getDb().then( function() {
            dfd.resolve();
        });
        return dfd.promise;
    }
}

controller.js

var Controller = module.exports = function(container) {
    this.container = container;
    this.db = container.getDb();
}

Controller.prototype =  {
    indexAction: function(req, res) {
        this.db.all("SELECT * FROM testing",
            function(err, rows) {
                res.json(rows);
            });
    },
    ...
}

This looks like a good start for configuring action controllers with container based dependencies on a very low level. This concept can be extended to have a configurable container, to load a dependency tree and prepare singleton services, even to lazy load services using proxy objects. Et voila: welcome back to the good old Spring / Symfony world. You can write your code that way, it’s going to work pretty much as expected (I worked that way for quite some time)

Out of the rabbit hole

Wake up, Alice – you’re not in Wonderland anymore. Here’s what I’m doing in node.js these days; pretty, lean and simple.

/code

DB.js

var sqlite3 = require("sqlite3");
var db = new sqlite3.Database(":memory:");

db.run("CREATE TABLE testing " +
    "(id INTEGER PRIMARY KEY AUTOINCREMENT, " +
    "info TEXT)");

module.exports = db;

app.js

var express = require("express");
var http = require('http');
var Controller  = require('./controller.js');
var DB = require("./DB.js");

var app = express();
app.use(express.bodyParser());

app.set('port', process.env.PORT || 3000);

app.use(app.router);

app.get('/foo', Controller.indexAction.bind(Controller));
app.get('/foo/:id', Controller.fooAction.bind(Controller));
app.post('/foo', Controller.addAction.bind(Controller));

http.createServer(app).listen(app.get('port'), function(){
    console.log('Express listens on port ' + app.get('port'));
});

controller.js

var DB = require("./DB.js");

module.exports = {
    indexAction: function(req, res) {
        DB.all("SELECT * FROM testing",
            function(err, rows) {
                res.json(rows);
            });
    },
    fooAction: function(req, res) {
        DB.get(
            "SELECT * FROM testing WHERE id = $id",
            {$id:req.params.id},
            function(err, row) {
                res.json(row);
            });
    },
    addAction: function(req, res) {
        DB.run(
            "INSERT INTO testing (info) VALUES ($info)",
            {$info: req.body.info},
            function(err) { //this: statement
                res.json({ lastId: this.lastId});
            });
    }
}

The “magic” that might seem unusual to mature developers coming from class oriented environments lies in the “module” concept that’s one of node’s cornerstones. A module is the encapsulation of state and behavior, it can be used in a service-like fashion like I did in the last example. Notice, that I’m requiring DB.js in app.js without even using it. That way node.js executes the code inside once and keeps the reference in module.exports – the database therefore is prepared when a controller is using it. Well, not exactly: if the preparation / setup of resources takes really long, the application is up before initialization has finished (try wrapping the database in a timeout, I left the code as comment in the repo). But what’s more important: taking this approach you don’t have to take much care about your dependencies but rather can access them from any module you want simply by requiring them. The db variable in DB.js always refers to the very same instance.

tl/dr Lessons learnt: node.js and expressjs don’t propose a structure for managing dependencies and wiring them up with your clients. While you could write your code in a rather classical way, it’s mostly much simpler to use node’s builtin concepts.

PS. I just found this stackoverflow question that’s underlining the concept I tried to explain here in short. Good, that I’m not alone with my opinion 😉

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s