The story of Node.js reads like it came from a Hollywood script assembly line: Some kids are monkeying around with scrap they picked up around the Internet and find a new way to snap it together. The next thing you know, they're lapping the pack at the racetrack and coasting to the winner's circle.
In technical terms, the Node.js team, which is largely led by Ryan Dahl, began looking for a new way to serve up documents from a website without chewing up as much RAM as the traditional tools. They grabbed a copy of the V8 JavaScript engine to speed up their code, then created simple libraries that connected the V8 engine to the TCP/IP ports. Presto change-o, it was answering requests very quickly.
[ Also on InfoWorld: "JavaScript conquers the server" | Node.js is an InfoWorld 2012 Technology of the Year Award Winner. For more insight on software development, subscribe to InfoWorld's Developer World newsletter. ]
Once people noticed the speed, they started actually using the tool and the challenges began. It's easy to make a car by bolting together spare parts that are lying around, but it's an entirely different task to make something that can carry a load or stand up to the daily commute. The first version of Node ran from a command line containing the name of the file where the instructions were stored. That's not a great way to run a big, stable, professional website.
The scruffy, junkyard quality of the Node realm is rapidly disappearing as programmers build extras for Node as quickly as Node itself was born. Now Node is more of an ecosystem, with its own conference and a collection of tools that orbit around it. There are IDEs, deployment tools, and companies looking to offer Node hosting as a service. All are helping to transform Node from science experiment to real contributor in the data center and the cloud.
These tools aren't luxury goods yet. They're largely the first builds, which means there are rough spots and glitches. They are typically small, simple, and far from full-featured solutions, but they're also a chance for the Node users to rethink what did and didn't work with the previous generation of Web tools. Node is addressing some of the problems people have found with the traditional stack and fixing them.
They're also rediscovering some of the reasons why the previous generation of Web developers did many of the things they did. It's a bit of a stretch to say that Node exists because it can be so costly to allocate the RAM to start up a new thread, but the Nodesters are beginning to understand why the last line of programmers created the threads in the first place. There are already a number of angry posts about Node filled with four-letter words because the JavaScript callback paradigm is too unnerving.
To understand a bit of the excitement coming from this corner of the Net, I spent some time unpacking the Node tools and putting them through simple tasks. The more I experimented, the more I found. The area is growing quickly, and this summary is already incomplete as I file it. The good news is that the ground is incredibly fertile. Node.js has revealed that running a website doesn't need to be that complicated. The tools it is spawning may not be perfect yet or ready for everything an enterprise requires, but they're real and they're beginning to play a role.
Node.js tools: Cloud9 IDEThe Cloud9 IDE is a development environment built into a website or, as the ad copy reads, development-as-a-service. You build your code with the Web app and the optional Chrome plug-in. Then, after testing it locally, Cloud9 will deploy the code into Heroku, the Joyent Cloud, or Microsoft's Azure (more on these clouds below).
The Cloud9 editor is sophisticated, providing more than the basic options available with a <textarea> field. Bolted-in extras include a JavaScript beautifier that will fix white space and clean up indentation. The code is colored, given line numbers, and semiparsed to identify undeclared variables.
Although many of the features won't surprise anyone using an IDE on their desktop, the surprise is that Cloud9 feels like a desktop tool in your browser. My favorite part is the command-line field where you can type in Git commands just as you would in your terminal window.
The service is not just aimed at Node programmers. You can also upload PHP and Ruby. The editor even claims the ability to colorize the syntax of C++, although I'm not sure what would happen to this code.
Everything here is free if you're willing to share your code with the world. If you want private projects and a private environment when it's available, the price is $15 per month. Cloud9 also promises to add collaboration tools.
Node.js tools: DreadnotOn the surface, Node looks easy to use in a product. Once you write your code, you move it to the machine and type node myprogram.js. The node runtime handles everything because you've coded many of the details into the JavaScript yourself. You choose the port for listening and the protocol, all in JavaScript code.
Alas, there are never enough configuration options in life, so serious Node folks created Dreadnot, a terrible pun, and set it to work organizing the Node stacks of code. It logs into your Git directory, checks out your code, and starts it up on your server. You control all of this with another configuration file, and Dreadnot handles the deployment. This is a nice way to begin automating a checklist of what needs to be fixed and checked before you run your code in production.
The Dreadnot code, now open source, was based on a tool called Deployinator created by Etsy to handle some of the workload around its craft marketing site. There's not much documentation, but that isn't a problem yet. Most of the options are self-explanatory, except perhaps the password file. But the code is all there and it works.
Node.js tools: Eclipse JavaScript debuggerOn the other end of the spectrum is the Eclipse IDE. There is now an Eclipse plug-in from the Google Chrome team that connects the venerable tool with the back end of the V8 JavaScript engine. If you start Node.js with the <tt>--debug</tt> option, Node will look to talk with any remote debugger. Google's Eclipse plug-in for debugging Chrome seems like just the ticket.
It's a nice idea, but I found it buggy. A decorator function kept clogging the stack of Eclipse, and nothing seemed to communicate as well as it could. While I could pause Node and poke around, I found it easier to debug Node using the output of the command line that started it. The Eclipse interaction was too much trouble.
These glitches are usually cosmetic. It's entirely possible that you won't experience them on your version of Eclipse or your version of Node. (I used Eclipse Indigo EE and 0.6.10.) My problems may also have been caused by all of the other plug-ins littering my Eclipse install.
Node.js tools: FeedHenryIt's not really fair to think of FeedHenry as just another Node hosting service -- it's much more. It's really a tool for building client/server apps that just happens to use Node underneath. You write your server-side code in JavaScript, and it runs on Node, even if that's not immediately apparent. FeedHenry emphasizes the advantage of using Node by including a shared directory for code that runs on both the client and the server.
There are many ways to construct an IDE for Node, and like Cloud9's, FeedHenry's is built for the browser. The development environment and your code sit in the FeedHenry cloud. You work through your browser, which also pops up an emulator.
The IDE is just the center of the universe because FeedHenry can build your app into stand-alone versions for a long list of mobile platforms. In addition to the most popular (iPhone, Android, RIM), FeedHenry supports other mobile platforms that aren't as well known: Opera WGT, Nokia Web Runtime. This cross-platform magic is possible because FeedHenry builds your application in HTML, CSS, and JavaScript.
Node.js tools: Nide"Nide" stands for "nice integrated development environment," and it's one of the simplest development environments I've seen. It's surprising, at first, how little is required to make it possible to create software, but then I keep forgetting what people did in the '70s and '80s with vi. We've become fat and spoiled.
Nide is a Web app that lets you browse through files and edit them in your browser. It is, as they say, a perfect example of the Node team eating its own dog food. When you save a file in the right directory, Node will find it and start serving it up. Then you check it in another tab on your browser. Node handles all the compilation -- if that's what you call what V8 does -- and most of the other packaging that IDEs normally do. That leaves Nide to focus on displaying and editing the files.
Debugging is the tricky part. You have to put in code for printing and use local tools like Firebug to watch the traffic move back and forth. The "d" in "Nide" doesn't stand for "debugger." Nide's big nonediting feature is a tool for browsing the packages installed by NPM (Node Package Manager). You can power this indirectly.
Is Nide really "nice"? Yes, but only if you're looking for something simple. If you want to step through your code, it won't help you.
Node.js tools: HerokuThe Heroku cloud is another to embrace Node with both arms. Heroku also hosts Java, Scala, Clojure, Python, and Ruby, although Node seems the most ideal for Heroku's approach, which is to put your process in a "Web dyno." This is sort of like a virtual machine, but with a better name that avoids any confusion with a real machine. You choose how many dynos you want, and the virtual stack will spin them up.
It's probably not fair to focus too much on Node when talking about Heroku because the most impressive aspect of the Heroku cloud is the sheer variety of features. There's a big collection of add-on services that range from databases (MySQL, PostgreSQL, Redis, MongoDB) to connections to APIs (Twitter, Amazon). It's a big ecosystem that's getting bigger, and Node is just a part of it.
Node.js tools: Joyent CloudNode creator Ryan Dahl works at Joyent, which would probably be sufficient endorsement of the company for many people. But as they say in the television ad game, "Wait, there's more." Joyent is building out its own cloud for hosting Node applications (and other software), pricing the machines by the hour just like everyone else these days.
Joyent offers free development machines for those who want to try out the platform. Just go to no.de, a domain name Joyent bought to show its commitment. Joyent also sells SmartMachines if you want to get an IP address and hang up a shingle in the DNS tables. The "smart" here refers to the flexible structure of the virtualization, which Joyent says makes more RAM and more CPU cycles available to the instances. To back this up, the company offers benchmarks that suggest you get 5 times more CPU computations, 14 times more disk I/O, and 3 times more memory I/O than a similarly priced machine in Amazon's EC2 stack. Naturally, your mileage may vary.
Most newcomers will start and stop their Joyent machines through the Web interface, which offers slick real-time graphs of the loads. If you want to automate the process, there's a JSON-driven API that can handle the configuration through a script.
The machines let you deploy your code via Git, the repository-based version-control mechanism that's becoming more and more standard. If you push your version of server.js, Joyent will do the rest. If you need root access, you can log in with an SSH key, but it's not clear to me that you'll ever need to. It helps to be familiar with Solaris because the Joyent SmartOS is based on this venerable tool. If you don't want a SmartOS machine, you can choose from Fedora, CentOS, Ubuntu, and Windows. The premier database is MongoDB, the latest favorite for Node users, but Riak and MySQL machines are also ready to run.
Node.js tools: Microsoft Windows AzureMicrosoft took one look at Node.js and obviously was smitten. Redmond put it to work immediately, giving it a place of honor in the Azure cloud alongside .Net, Java, and PHP. If you want Microsoft to host your Node server in their cloud, it will charge you by the minute at competitive rates. It's all ready.
The Azure toolkit comes with a large collection of tools and so-called cmdlets that let you debug your server.js file on your own machine. When it's ready, you push it right into the cloud. Microsoft's tools for your machine are largely driven by the command line. You type long commands like Join-AzureNodeRoleToMongoRole, and all local configuration issues are handled for you. You don't need to remember the details. I'm sure Microsoft is working on a push-button application. Soon you won't even need to type all of the commands.
Azure's Node tools include most of the latest Node goodies like the connection with MongoDB, one of the more popular NoSQL databases. You can install these packages with NPM, then accelerate the development by letting Microsoft's cmdlets massage the XML in the configuration files.
The integration is still fairly preliminary. While all of the Node packages are available through NPM, you'll have to make some connections by hand to important options like SQL Server. Some people use the REST interface to SQL Server that Microsoft makes available. Although MongoDB is a nice option, Azure offers a number of reporting tools that are integrated directly with SQL Server.
I found that the Azure tools and documentation were quite nice but far from perfect. Microsoft brings a polish that's rare in the world of experiment where Node.js began. It clearly wants to capture Node users with the Azure cloud. Still, I found strange glitches that I couldn't get around. The local emulator would keep popping up alert windows at times, and they made it hard to debug.
Microsoft's investment in Azure is already substantial, and the decision to include Node shows that the tool is drawing plenty of interest in the corporate world. I think the technology holds a great deal of promise in the world of shared virtual servers because it's designed to minimize how much RAM is consumed, a premium in this realm. It would be fascinating to see a thorough study on whether Node is dramatically cheaper for Azure owners based on the consumption of resources.
In theory, you should be able to do more with Azure's smaller servers when running Node.js. The test versions start off using "extra small" instances, which are only 4 cents per minute. The longer you can hold off consuming larger machines, the cheaper your jobs become.
Node.js tools: NodejitsuThe cloud services from Nodejitsu aren't available for users yet because they're still in private beta. However, Nodejitsu is already releasing its tools under a generous open source license, so it's possible to poke around and see what the company is building.
The major tools simplify the process of creating and deploying Node applications to the cloud. The core piece is Jitsu, a command-line tool for juggling Node applications that's similar to Microsoft's Azure tools. You type one command and a new application is created. Type another command and the application starts up for testing. Type yet another and the application will be pushed to the Nodejitsu cloud (coming soon).
If you want to build your own cloud, you can use Haibu, a local application server that allows Node applications to capture as much of the CPU as they might need. Haibu will wrap your application with a layer of code called a "carapace" and turn them into "drones" -- Nodejitsu's term for the copies floating around the cloud.
Nodejitsu is pushing the Node community forward with these tools, and the open source license makes them attractive for companies that want to experiment without being locked into the Nodejitsu cloud.
Reinventing the Web stackThere's plenty of good news to report. The Node.js ecosystem is growing at an exponential rate, and there's a bazillion times more code that's ready to use and enjoy. Of course, it's only natural at this point in time because it's easy to grow quickly when you start with nothing. These tools are building a sustainable infrastructure that's already doing useful work in small corners of the enterprise world.
It's easy to predict that there will be even more advances coming very soon and to imagine new tools that shouldn't be hard to build. I'm sure we'll soon start seeing versions of the IDE that are tightly integrated with the website itself. Many of the Web content management systems such as Drupal have little switches that appear when an administrator logs in. If you want to change a part of the site, clicking on the nearest switch will take you to a screen where you can simply edit some PHP and hit Save. There's no need to fire up a text editor or sift through a pile of files for the source code. Everything is done right on the browser.
This kind of development environment would be ideal for Node, if only because the structure of the framework is even more disjointed than the typical programming language. Node's nested callback structure lends itself to a form-based tool that would have separate sections for code that's executed before and code executed after. I find the callback structure maddening and difficult to work with, especially when it's several levels deep. A nice UI into these depths would simplify matters dramatically. Only geniuses can manage to count all those nested brackets.
An innovation like this would go a long way to fixing some of the biggest problems that people are reporting with Node. You'll have no trouble finding developers' blogs filled with four-letter words because they hate the "spaghetti callback" design paradigm. People can manage this when the site is a simple layer on top of some database, but they go nuts when the application gets more complicated.
The websites of cloud providers like Joyent also show how a good UI will make it easier to work with Node. The instrumentation is much better than that offered by the basic download, and it's probably going to get better as the company invests more into making its SmartMachines a platform that sustains enterprises.
We're sure some serious folks will complain that all of this work is just re-implementing many of the features that Ruby and Java programmers already take for granted. They're right. The Node ecosystem still needs plenty of work before it can begin to compete with the traditional standard bearers that keep the serious sites running. Although much of this new work has some of the polish that serious software should carry, the entire ecosystem still has the feeling of a really, really good project for science class.
If you're a serious developer with a boss who demands real promises of performance, you can wait a year or two before tuning in again. It's still not worth your time. But if you're interested in experimenting and watching the tools evolve, you'll find plenty of interesting software to fill an afternoon or three. There's even enough to drive a simple Web service or database front end if you want to give it a real try.
This article, "Node.js tools: Server-side JavaScript comes of age," originally appeared at InfoWorld.com. Follow the latest news in programming and open source at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.
Read more about application development in InfoWorld's Application Development Channel.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.