animesh kumar

Running water never grows stale. Keep flowing!

IBM has no idea what Node.js is

with one comment

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

People, don’t read this article on IBM’s developer works Just what is Node.js?. And you have read it already, don’t believe it. IBM has no idea what Node is, really.

Read my rebuttal http://wkp.me/wkk6g

For more gory details, read Marak Squires’ blog here.

IBM must pull the article or ask the author to rewrite it. The article is providing a dis-service to any new developers who might stumble along it as their first introduction to node.js.

Advertisements

Written by Animesh

May 13, 2011 at 12:34 pm

Node and Its Many Incarnations (Node Version Management)

leave a comment »

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

Node.js is under active development. And every other day, a new build is released. It’s awesome to see how fast Node is growing and how vibrant the community is… but on the down side, it’s becoming increasingly difficult to keep track of its many versions, and API changes.

Very often, while developing an app, you find yourself married into a particular Node version, because a newer one might have some API changes (mind you, Node is witnessing heavy transformations, especially at the API level) which might break you app… and then, you would be forced to revert back to the older version. That means, uninstall the current node and re-install the older one. Ouch! So much work for a mere upgrade.

Well, there is a nicer way to do it. Check out this project by Tim Caswell: Node Version Manager. It does exactly what it says. It manages various Node versions on your machine, development, stage, production whatever. How?

It creates a virtual Node environment for each version you want to keep. Let’s say, you want to stay with the last stable release v0.2.6 (from the time you started your app) but also want to experiment with v0.4.7 to keep an eye on new additions.  NVM will install two separate Node(s) for you, and each will run in its own sandbox like environment, that is, you will have to install all your third party Modules/Libraries separately for each Node installation. That might seem to be a lot of work, but trust me, it’s the safest way to avoid conflicts. Okay. Let’s get to work.

Installation

Note: I am assuming that you have basic knowhow of GIT (the most awesome source control management system).

  1. Clone NVM repository to your local machine:
    $ git clone git://github.com/creationix/nvm.git ~/.nvm

    Above command would close the NVM repository to a folder ‘.nvm’ in your home directory. (I am using Ubuntu 10.0.4)

  2. Switch to folder ‘.nvm’ and make file ‘nvm.sh’ executable:
    $ chmod 755 ~/.nvm/nvm.sh
  3. ‘nvm.sh’ is just a shell script, so in order to run it, you must source it to every terminal you open. To do this automatically, simply edit either ‘.bashrc’ or ‘.profile’ file to have this line in the very end:
    . ~/.nvm/nvm.sh
  4.  That’s it. Open a new terminal and run,
    $ nvm
  5. You will see a set of useful commands you can use. 🙂 Easy huh?

Getting dirty

Before you get any further, just make sure that you have ‘wget’ installed in your machine. I know, I know… you might already have it. I just want you to make sure.

Check which versions of Node are available.

$ nvm sync // update the local machine with available versions from server
$ nvm ls   // displays all available and installed versions

Now install Node v0.4.7.

$ nvm install v0.4.7 // will install Node v.0.4.7

Note: You might get this error, “Could not autodetect OpenSSL support. Make sure OpenSSL development packages are installed. Use configure –without-ssl to disable this message” which says, that you need to install SSl library:

$ sudo apt-get install libssl-dev

NVM creates a folder ‘src’ either in your home directory or in ‘.nvm’ directory where it downloads the bundled release, extracts and installs it. NVM also installs NPM (node package manager) for each installation of Node.

Select a particular version

$ nvm use v0.4.7 // start using Node-v0.4.7

That’s it. You have set up a system which will enable you to quickly and cleanly switch between various Node versions. You can test your app’s compatibility with any of them, and if need be, easily switch to the one your app was most comfortable with.

Now, since you have set up a congenial Node development machine, in the next blog, I will talk about how to go live with your Node app.

Note: for CentOs-5.x, please make sure that you have following packages installed:

$sudo yum install gcc-c++ screen git-core openssl openssl-devel

Written by Animesh

May 3, 2011 at 11:17 am

shutdown hooks in java

with one comment

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

You know what? You can add your custom hooks to JVM to be called when JVM shuts down. Interesting… no?

What happens is very simple: you create a Thread and write you logic in there, and then register this Thread to the Runtime instance. Here is a sample:

public class JVMShutdownTest {

	public static void main(String[] args) {

		// Add a shutdown hook
		Runtime.getRuntime().addShutdownHook(new Thread(){
			public void run(){
				System.out.println ("closing down the shop...");
			}
		});

		// Exit now
		System.exit(0);
	}
}

output will be: closing down the shop…

This attached Thread is initialized but not yet started. When JVM starts to shut down, it starts all registered hooks in an uncontrollable (that is, there is no way to enforce your will) order, and all of them run concurrently.

One thing you must take in consideration is that these hooks get executed at a very delicate time and so you must keep them light, thread-safe and independent of heavy dependencies. Thumb rule: hooks must finish quick.

Read more >>

Written by Animesh

March 28, 2011 at 4:41 pm

Posted in Technology

Tagged with ,

WebSocket support in Android’s Phonegap apps

with 79 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

We are developing a small game which can be played from multiple users using variety of clients, web-browser, Android, iPhone, iPad etc. It’s like, there is a server and all clients connect to this server, and send and receive messages. We decided to use WebSocket for underlying connection between clients and server, and Phonegap to build clients. Our idea is to write the client once and then run it on variety of platforms. Since, Phonegap enables app development using HTML, CSS and JavaScripts, it generously fits into our requirement.

But Phonegap doesn’t support WebSocket yet, it’s in their Plan-of-Action for 1.x release though. So, it was needed to be addressed. I found Mathias Desloge’s PhoneGap-Android-HTML5-WebSocket project. It was good but it used old java.io.* packages. I would have preferred to use java.nio.* for better and efficient non-blocking behavior. So, I decided to write my own small library.

Library can be found here: websocket-android-phonegap.

How to use?

  1. Copy Java source into your source folder.
  2. Copy websocket.js in your assets/www/js folder
  3. Attach com.strumsoft.websocket.phonegap.WebSocketFactory to WebView, like
    	@Override
    	public void onCreate(Bundle savedInstanceState) {
    		super.onCreate(savedInstanceState);
    		super.loadUrl("file:///android_asset/www/index.html");
    
    		// attach websocket factory
    		appView.addJavascriptInterface(new WebSocketFactory(appView), "WebSocketFactory");
    	}
    
  4. In your page, create a new WebSocket, and overload its method ‘onmessage’, ‘onopen’, ‘onclose’, like
    	// new socket
    	var socket = new WebSocket('ws://192.168.1.153:8081');
    
    	// push a message after the connection is established.
    	socket.onopen = function() {
    	 alert('connected');
    	};
    
    	// alerts message pushed from server
    	socket.onmessage = function(msg) {
    	 alert(JSON.stringify(msg));
    	};
    
    	// alert close event
    	socket.onclose = function() {
    	 alert('closed');
    	};
    

How it works?

When you create a new WebSocket object in your page, behind the scene, websocket.js delegates the responsibility to com.strumsoft.websocket.phonegap.WebSocketFactory to instantiate new com.strumsoft.websocket.phonegap.WebSocket object.

		// websocket.js
		// get a new websocket object from factory (check com.strumsoft.websocket.WebSocketFactory.java)
		this.socket = WebSocketFactory.getWebSocket(url);

WebSocketFactory simply instantiates a new WebSocket object, connects it to the designated server and returns the instance.

// com.strumsoft.websocket.phonegap.WebSocketFactory

public WebSocket getWebSocket(String url) throws URISyntaxException {
	WebSocket socket =  new WebSocket(appView, new URI(url));
	socket.connect();   // connects to server
	return socket;
}

Now, whenever an event occurs, say, ‘onmessage’, WebSocket class delegates that event to Javascript.

// com.strumsoft.websocket.phonegap.WebSocket

public void onMessage(String message) {
	appView.loadUrl(buildLoadData("message", message));
}
private String buildLoadData(String _event, String _data) {
	String _d =  "javascript:WebSocket.on" + _event + "(" + 
				"{"
				+ "\"_target\":\"" + webSocketId + "\"," + 
				"\"_data\":'" + data + "'" + 
				"}" + 
			")";
	Logger.log(_d);
	return _d;
}

Finally, ‘WebSocket.onmessage’ from websocket.js is called. It parses the payload, finds out the target WebSocket object, and calls the corresponding event on the target object with event data.

	// websocket.js
	// static event methods to call event methods on target websocket objects
	WebSocket.onmessage = function (evt) {
		WebSocket.registry[evt._target]['onmessage'].call(global, evt._data);
	}

That’s it!

Amendment

(Date: Thu Aug 25 12:40:52 IST 2011)

There was a serious bug! The Websocket connection runs in a separate thread to manage persistent state with the server. And the front end Javascript (websocket.js) stays within UI/Main thread. And Android doesn’t want other threads to communicate with UI thread directly. These threads must employ an additional thread to bridge the communication. So, here is the fix!

	// a message is sent to server! 
	public void send(final String text) {
		// new thread
		new Thread(new Runnable() {
			@Override
			public void run() {
				if (instance.readyState == WEBSOCKET_STATE_OPEN) {
					try {
						instance._send(text);
					} catch (IOException e) {
						instance.onError(e);
					}
				} else {
					instance.onError(new NotYetConnectedException());
				}
			}
		}).start();
	}

	// when a message is received
	public void onMessage(final String msg) {
		// post a new thread to View
		appView.post(new Runnable() {
			@Override
			public void run() {
				appView.loadUrl(buildJavaScriptData(EVENT_ON_MESSAGE, msg));
			}
		});
	}

Commit link:
https://github.com/anismiles/websocket-android-phonegap/commit/a7ccb815cce3a446c3ec92058187cdb20e5a41e8
https://github.com/anismiles/websocket-android-phonegap/commit/087f7a93d46f92cb037d2b451a4d253a65f5f015

Written by Animesh

February 3, 2011 at 11:52 am

Posted in Technology

Tagged with , , ,

WebSocket and node.js: why shud’ya care?

with 26 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

Traditional HTTP messages are heavy. Every message is sent with HTTP headers. Now, let’s say you have an application that has a real-time component, like chat or some twitter client or may be some traffic analysis stuff. And let’s say you have around 100,000 users connected to your app. To make your app real-time, you need to have a mechanism which will enable server to push data almost as soon as this data becomes available. You could do it in two ways: Write a script which will connect to server every few seconds to check if there is any data. With each attempt, full set of HTTP headers moves back and forth between client and server. That’s not very efficient. To save yourself with all these bandwidth hassles, you could use a popular trick known as long-polling, where your browser connects to server and server holds the connection open until there is some data available to be pushed.

Now, let’s assume that there are 100,000 users connected to your app and every 10 seconds some data is sent from server to clients. Following HTTP specs, every time some data is sent, full set of headers are shared between client and server. This is how they look,

Request header

GET / HTTP/1.1
User-Agent: ...some long user agent string...
Host: animesh.org
Accept: */*

Response header

HTTP/1.1 200 OK
Date: Tue, 25 Jan 2011 17:32:19 GMT
Server: Apache
X-Powered-By: PHP/5.2.3
X-Pingback: http://animesh.org/endpoint/
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

That’s approximately 350 bytes of data, per user every 10 seconds. That’s roughly 28,400,000 bits per second of network throughput for 100,000 users. Roughly 26.7 Mbps for only HTTP headers. Gosh!

WebSocket

WebSocket comes to resue. With web sockets, once a handshake is done between client and server, messages can be sent back and forth with a minimal overhead. That’s awesome. You do a handshake while establishing the connection, and of course handshaking needs all those HTTP headers, but after that, you only need to send the data… no headers. This greatly reduces the bandwidth usage and thus improves the performance. Let’s see how. This is how handshake headers look like,

Handshake Request header

GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: animesh.org
Origin: http://animesh.org
WebSocket-Protocol: sample

Handshake Response header

HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: http://animesh.org
WebSocket-Location: ws://animesh.org/
WebSocket-Protocol: sample

And now, the connection has been established and data can freely flow between server and client without having to exchange any HTTP headers until this connection is closed or broken and you do another handshake. Imagine how much bandwidth you are saving! Whoa!

Example

Let’s write a simple application to see and learn how this thing actually works. This application will have a server all the clients will connect to, and whenever one client writes something to the server, all clients will be notified.

Here is our server, written in Node.js. Let’s name it server.js

Note: Though you can very well write a web socket server using Node’s native APIs, however I chose to use Micheil Smith‘s node-websocket-server library. This library is simple, elegant and very easy to work with. It works by wrapping and extending Node’s server object.

var sys = require("sys");
// Library https://github.com/miksago/node-websocket-server
var	websocket = require('./lib/node-websocket-server/lib/ws/server');

// create web socket server
var server = websocket.createServer();
// listen on port 8078
server.listen(8078);

// when the server is ready
server.addListener("listening", function() {
  sys.log("Listening for connections on localhost:8078");
});

// when a traditional HTTP request comes
server.addListener("request", function(req, res) {
	res.writeHead(200, {
		"Content-Type" : "text/plain"
	});
	res.write("This is an example WebSocket server.");
	res.end();
});

// when a client websocket connects
server.addListener("connection", function(conn) {

	// when client writes something
	conn.addListener("message", function(message) {

		// iterate thorough all connected clients, and push this message
		server.manager.forEach(function(connected_client) {
			connected_client.write(JSON.stringify(conn.id + ": " + message));
        });
	});
});

Now, let’s write a simple client. We will create one HTML file and run it in Google Chrome. Let’s name is client.html

<!DOCTYPE html>
<html>
<head>
    <title>WebSocket - Simple Client</title>
    <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.js"></script>
    <script type="text/javascript">

	$(function() {
		// bind form
		$('#payload-form').submit(function() {
			var payload = ($("input#payload").val());
			socket.send(payload);  // write to server
			return false;
		});

		// open websocket
		var socket = new WebSocket('ws://localhost:8078');

		socket.onopen = function() {
	    	// Web Socket is connected. send an initial random message.
	    	socket.send(Math.floor(Math.random()*11) + ' >> Hi, I am Mr. so-and-so!');
	    };
		// append to '#log' whatever server pushes.
		socket.onmessage = function(ev){
			msg = JSON.parse(ev.data);
			$('#log').append(JSON.stringify(msg) + '</br>');
		}
	})
    </script>
</head>
<body>
	<div id='payload-container'>
		<form id='payload-form'>
			<input type='text' name='payload' id='payload' value='Hello World' style="width:500px;"/>
			<input type='submit' value='push'/>
		</form>
	</div>

	<div id='log' style='display:block; border:1px solid lightgray;'></div>
</body>
</html>

Now, run your server, and open your client in multiple Chrome windows/tabs.

// run server
$ node server.js

That’s it! Was is fun? I will write more on how to establish WebSocket connections from a Java client in the next blog.

Written by Animesh

January 25, 2011 at 3:25 pm

i’m a joke

with one comment

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

I’m a joke
may be, a laugh.

Don’t trust me
not worth it
I come cheap
and easy
like wild scrubs
at every nook
and cranny.

Never taken home
but, used
and cast away
and laughed at
worst, ridiculed sometimes
I’m not worth it.

Written by Animesh

January 19, 2011 at 3:29 pm

Posted in Poetry

Tagged with , , , , ,

Maven, SQL and ordered execution

leave a comment »

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

I have been re-architecting this new project. This is huge. And broken. And I was called upon to clean, refactor and re-architect it. Phew!

I thought a good strategy would be to deal with it in outside-in style, which is, fixing the build part first then moving down to various parts. One of the interesting things that they had done was using Maven to populate PostgreSQL DB Schemas and then load data into the DB.

Steps of execution

  1. Drop everything
  2. Create users
  3. Create schemas
  4. Load data
  5. Load upgrade data

They used sql-maven-plugin underwhich they had multiple execution tasks. The whole thing was very untidy and I didn’t like it.

Summary

  1. They used <fileSets> to import sql files. FileSets are an elegant choice if you want to import files with ANT like wildcard patterns. But the files are alphabetically sorted. So, unless you don’t care about the order in which the files will be executed, you should avoid using FileSets. In their case, they wanted to run ‘drop_users_databases.sql’ before ‘create_users_databases.sql’. With FileSets, it was impossible to do, they would be re-ordered just the opposite. Solution? They should have used <srcFiles>, but instead they created 2 separate <execution> tasks to run in order. Too much verbosity. Eh?
  2. Another issue was with loading upgrade data. Now, they follow agile methodologies and keep updating their DB schemas, and so after few iterations, before they merge the changes in the main script, they end up with a number of files like upgrade-schema-2.0.4.sql, upgrade-schema-2.0.5.sql, upgrade-schema-2.0.5.1.sql and so on. Pain is, they don’t have any automated mechanism to execute these files in order, so the developers would be forced to do it manually. If the number of these files were low, that wouldn’t have been a problem… but how about thirty or forty such files? Pissed off.

Strategy

  1. Kick <fileSets> out of the window
  2. Use <srcFiles> instead (because I cared more about order)
  3. Logically partition the operations
  4. Create a new profile to help run these DB executions at will

Code

<profiles>  
<profile>  
	<!-- profile id -->
	<id>init-db</id> 
	<build>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>sql-maven-plugin</artifactId>
				
				<!-- postgresql dependencies -->	
				<dependencies>
				  <dependency>
					<groupId>postgresql</groupId>
					<artifactId>postgresql</artifactId>
					<version>8.3-603.jdbc4</version>
					<scope>clean</scope>
				  </dependency>
				</dependencies>
				
				<!-- DB Configuration -->
				<configuration>
					<url>jdbc:postgresql:my_db</url>
					<driver>org.postgresql.Driver</driver>
					<username>user</username>
					<password>pass</password>
					<autocommit>true</autocommit>							
				</configuration>
				
				<executions>
					<!-- 1. drop everything, create users and load data -->
					<execution>
						<id>drop-and-create-database</id>
						<phase>clean</phase>
						<goals><goal>execute</goal></goals>
						<configuration>
							<srcFiles>
								<srcFile>src/main/sql/drop_users_databases.sql</srcFile>						
								<srcFile>src/main/sql/create_users_databases.sql</srcFile>						
								<srcFile>src/main/sql/create_schema.sql</srcFile>						
								<srcFile>src/main/sql/load_data.sql</srcFile>						
								<srcFile>src/main/sql/test_data.sql</srcFile>						
							</srcFiles>    															
						</configuration>
					</execution>			
					<!-- 2. run all upgrade scripts -->
					<execution>
						<id>upgrade-schema</id>
						<phase>clean</phase>
						<goals><goal>execute</goal></goals>
						<configuration>
							<srcFiles>
								<srcFile>src/main/sql/upgrade-schema-2.0.6.sql</srcFile>						
								<srcFile>src/main/sql/upgrade-schema-2.1.5.sql</srcFile>						
								<srcFile>src/main/sql/upgrade-schema-2.1.7.sql</srcFile>						
								<srcFile>src/main/sql/upgrade-schema-2.1.8.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-2.2.1.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.0.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.1.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.2.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.3.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.4.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.5.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.6.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.6.1.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.8.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.9.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.10.sql</srcFile>
								<srcFile>src/main/sql/upgrade-schema-3.0.11.sql</srcFile>
							</srcFiles>    						
						</configuration>
					</execution>
				</executions> 
			</plugin>
		</plugins>  
	</build>  
</profile>  
</profiles>  	  

How to run?

Note that I have bound the executions with phase ‘clean’ within a profile with id ‘init-db’. So, I just need to let maven know about the phase and profile, like this:

mvn -Pinit-db clean

I hope this will help someone in need. I will post more adventurous code cleaning stuffs as I will encounter them. Cheers!

Written by Animesh

December 28, 2010 at 12:31 pm