Saturday, March 2, 2013

Appending text to a file in Node.JS

Recently I've been testing the application I'm working on. Since it is already complicated by itself I don't want to add a database support to it. Instead, I thought about writing data on a file. In the context of my software, I'm sending messages to different processes on remote machines and I want to be sure I'm not loosing any of those messages. To check this, when I receive a message I store its ID (every message has one) in a file, appending it to the end. Then by using the sort bash command I sort them and see which messages I lost (or simply through a wc -l filename I can check that all the messages I sent arrived).
To write on a file in Node.js this is the usual process:


var fs = require('fs');
//[...]
fs.writeFile('log.txt', 'Hello', encoding='utf8', function (err) {
    if (err) throw err;
});


But this would overwrite the content of the file and write 'Hello' in it. To append text on a file, recently Node added the following command to their APIs:


var fs = require('fs');
//[...]
fs.appendFile('log.txt', 'Hello', encoding='utf8', function (err) {
    if (err) throw err;
});


As you can see it works exactly like the writeFile function except that this would append 'Hello' at the end of the file. Moreover, if the file doesn't exist yet, it will create it for you. Pretty useful!
What I noticed while looking at the file that was filled in the meantime with IDs (I kept inspecting it through wc -l filename) is that sometimes it "blocked" itself on some values, and then after a while it added a bunch of IDs and then again it blocked, and so on. I was pretty frustrated by this, since I thought it was a problem of my topology. In the end I asked Google about the appendFile function, since I was interested in its performances and I found out that it is asynchronous. That means I never had any error concerning the odd behaviour it had, it was just because the execution of the function itself is asynchronous. Thus I searched for a synchronous way to append text on a file and on the Node API page I found this:


var fs = require('fs');
//[...]
fs.appendFileSync('log.txt', 'Hello', encoding='utf8');


As you can see it works a little bit differently from what appendFile does: it doesn't take a callback function anymore. And that's it. Now it appends correctly each time the process receives a message, so I can better monitor what's going on.
Categories:

4 comments:

  1. Hello,

    I had the same problem (needing to persist data without having to install a full-blown database) so I wrote a pure javascript database that can persist to a file (using the appendFile function). It provides the most common subset of MongoDB's API, for clean CRUD operations: https://github.com/louischatriot/nedb

    ReplyDelete
  2. Thanks for the pointer, looks awesome!

    ReplyDelete
  3. @Masiar Thanx

    ReplyDelete