Holy sh$% Batman: showing async blocks of data from multiple sources with jQuery, Node JS and Express JS and the problem with Virtuals, DynamicHelpers
I’ve been using Node.js lately, and it has some problems like everything on planet Earth because it’s not perfect. I’ll give you an example and my solution:
You have a page, that shows a list of something that comes from the database. This is just main content, could be a list of blogs or something. On the sidebar you have totally unrelated (or related) content that also comes from the database. The problem with Node.js (especially using Express JS) is that you have to build the render object before you pass it to res.render which in turn will pass it to the template file (jade or whatever). I’ll explain the two ways of doing this:
Using async
Remember you have to query the database to get “somedata” and “somextradata” to pass both of them to res.render:
res.render(“myview”, {listone: somedata, listtwo: someextradata});
Using async you’ll do something like this:
Easy. You just executed two queries in series, first one, then the other one, and when both finish, you execute a last callback which is gonna have a results object with the data from each of the callbacks you executed in series. With that object (the results object) you can do whatever you want, like passing it to res.render(“myview”, results);
The problem with this approach is that you don’t really have lots of reusable code and you have to wait until all queries finish.
Using callback hell
Love it or hate it, callback hell works like this:
What happened there? We first do the first query, save the results in the var someData and after we have the results we do the second totatlly unrelated query, when we have the results from the second query we save it in the var someExtraData. Since we only have two queries, we build an object “results” that includes someData and someExtraData and we pass both results to callback or even to res.render(“myview”, results)
The problem is the code, no wonder why they call it callback hell. And again, both ways are async. So, you can’t show the page to the user unless everything is finished building.
But, again, back to the original question, what if someExtraData is just a meaningless peace of content, that kind of content you use just to populate the right side of your page? You don’t really want to have all those problems do you? And what if it is dynamic? What if you don’t have to actually wait for the first query to return?
Well, I’m still trying to find a good approach to solve this problem, in the meantime I’m using jQuery. Yes, jQuery.
This is my approach:
First, build the important data into the main content area like you would do if you didn’t need someExtraData.
Then, create another app.get route to fetch only the someExtraData. Something like this
With that you can query http://example.com/someExtraData and get a json object. This will be in a total different time from the main content. And with jQuery you just do:
Now, this approach works, and I think is pretty good, if you have any comments please let me know here will you?
Partials and dynamicHelpers (locals in 3.0)
Finally I don’t want to end without talking about those guys.
Partials are a thing of express/jade (don’t really know) in which you can reuse small peaces of jade code to do things you always do, the best thing is that it uses dynamic placeholders so you can pass inside jade a variable to the partial.
Something like this:
partial(“myMenu”, {menu: items})
Items will be a full collection of menu items. Inside the myMenu.jade partial file, you’re supposed to render “items”. The problem with partials is that partials need data but this data always come from the main .jade file, which in turn, comes from the same source. So, both the main.jade and partial.jade files use the same data source: res.render(“main”, {myData: myData})
Now, dynamicHelpers or locals in Express 3.0 are a different thing, those are like “global” variables. Let’s say you print the version of your site on every .jade file you have. You don’t want to do:
res.render(“somefile”, {version: getVersion}
everytime do you? Instead, you use dynamicHelpers to process the getVersion, and inside all your jade files you’ll have always #{version}. What it’s also nice is the fact that you have access to req and res arguments.
Virtuals
Virtuals in Mongoose are a special property of the object you query, the property doesn’t exists in your model, you create it “on-the-fly”. Let’s say you have a firstName and a lastName in your model, and you always want to use the fullName, the problem’s you don’t have it, you need to build it using firstName and lastName. You do that with virtuals.
The problem with virtuals and dynamicHelpers are almost the same, the async problem. Node.js won’t wait till dynamicHelpers/Virtuals query the database. Node.js already rendered the page to the user by the time you’re thinking about querying the database. So, whatever you query, it won’t show up in your page.
I will post the solution at Github: https://github.com/lelizondo/nodejs-async-blocks-example
So, what do you think?