animesh kumar

Running water never grows stale. Keep flowing!

Posts Tagged ‘Java

Jesque-Guice binding

with 2 comments

In our Java stack, we use Guice IOC library quite heavily. We are kind of smitten with it. Now, when we needed a task queue to process our background jobs, we settled with Jesque which is a port of Github’s Resque project, and runs on Redis which was already part of our stack. In the process, we also added delayed task functionality to this.

Problem

Anyways, the problem we were facing was with Injecting Guice dependencies into Jesque workers. Our workers performed heavy operations and at times had to read-write from DB, or speak with other services.

Jesque uses Java reflection APIs to instantiate and run its workers (which are Runnable classes) and so, it becomes very difficult to inject Guice managed objects and services into Jesque workers.

We started the hacky-and-ugly way by creating static references to relevant services that our workers needed. if it were for few services, this would have worked, but our workers kept growing in features and reach, and after a while the whole code base was stinking. We had to do something about it. Something elegant!

Solution

So, we created Jesque Guice binding project. You can annotate your worker classes, and Guice will then discover, register and start them. Let me show you some code.

First, let’s create an ExampleWorker. You have to remember that the Worker must implement Runnable interface, and it must be annotated with @Worker which Guice uses for discovery and binding.

// ExampleWorker

@Worker(job = "ExampleJob",                 // job name
        queues = { "EXP_QUEUE" },           // queue names
        enabled = true,                     // enabled
        count = 1,                          // 1 instance of this worker running
        events = { WorkerEvent.JOB_SUCCESS, WorkerEvent.WORKER_START }, // Events to listen to
        listener = EchoListener.class      // WorkerEventListener
)
public class ExampleWorker implements Runnable {
    // LOG
    private static final Logger LOG = LoggerFactory.getLogger(ExampleWorker.class);

    // Note: Only field level injection would work!
    @Inject
    ExampleService service;

    String arg1;
    String arg2;

    // Must keep an empty constructor for Guice to discover this
    public ExampleWorker() {
    }

    public ExampleWorker(String arg1, String arg2) {
        this.arg1 = arg1;
        this.arg2 = arg2;
    }

    @Override
    public void run() {
        LOG.info("Running worker={}, with arg1={}, arg2={}", new Object[] { getClass(), arg1, arg2 });
        // calling service
        service.serve();
    }

}

@Worker attributes let you control the behavior. The above worker, ExampleWorker, listens to EXP_QUEUE queue, accepts Jobs by name ExampleJob, has an WorkerEventListener defined by Guice Managed class EchoListener which listens to WorkerEvent JOB_SUCCESS and WORKER_START. There is only ONE instance of this worker running.

Note: ExampleWorker has been field @Injected with ExampleService. Please remember, that
Only field @Inject will work with Workers, because Jesque uses constructors to pass Job arguments.
– Also, you must keep an empty constructor around so as to let Guice discover this worker.

Now, since we have added a Listener, we must define it.

// EchoListener

public class EchoListener implements WorkerListener {
    // LOG
    private static final Logger LOG = LoggerFactory.getLogger(EchoListener.class);

    @Override
    public void onEvent(WorkerEvent event, Worker worker, String queue, Job job, Object runner,
            Object result, Exception ex) {
        LOG.info("onEvent ==>> queue={}, event={}", new Object[] { queue, event });
    }

}

For the sake of demonstration, this has been kept very minimal. But, mind you, you can @Inject any Guice managed objects into this, using constructor and/or field injection.

Now, let’s define the ExampleService that we want to @Inject into our worker.

// ExampleService

public class ExampleService {
    /** The Constant LOG. */
    private static final Logger LOG = LoggerFactory.getLogger(ExampleService.class);

    public void serve() {
        LOG.info("Heya! I am not here to serve, just to show you how injection works.");
    }
}

Wonderful! Let’s now bind these all together in a GuiceModule.

// ExampleModule

public class ExampleModule extends AbstractModule {

    @Override
    protected void configure() {
        // Jesque Guice
        install(new JesqueModule());

        // Jesque Client
        Config config = new ConfigBuilder().withHost("localhost").withPort(6379).withDatabase(0).build();
        bind(Config.class).toInstance(config);
        bind(Client.class).toInstance(new ClientImpl(config));

        // Worker
        bind(ExampleWorker.class).asEagerSingleton(); // Must be singleton
        // WorkerEventListener
        bind(EchoListener.class).in(Scopes.SINGLETON);
        // Worker Executor (This is where they actually run)
        bind(WorkerExecutor.class).to(SimpleThreadBasedWorkerExecutor.class);
        // Service (will be injected into workers)
        bind(ExampleService.class).asEagerSingleton();
    }

}

Here, first we install JesqueModule, and then bind other objects.
– Bind Jesque config and client.
– Bind Worker, Lister, Service etc.

You will notice we have also bound WorkerExecutor. This interface accepts net.greghaines.jesque.worker.Worker instance and runs that on a thread. Jesque-Guice comes with 2 simple implementations:

1. SimpleThreadBasedWorkerExecutor which run each net.greghaines.jesque.worker.Worker on an unmanaged separate thread, and
2. CachedThreadPoolBasedWorkerExecutor which creates a CachedThreadPool where net.greghaines.jesque.worker.Worker is run.

You can implement your own strategy or provide your own ExecutorService.

Run

Now we have everything, let’s run it then.

// Main

public class Main {
    /** The Constant LOG. */
    private static final Logger LOG = LoggerFactory.getLogger(Main.class);

    /**
     * @param args
     */
    public static void main(String[] args) {
        Injector injector = Guice.createInjector(Stage.DEVELOPMENT, new Module[] { new ExampleModule() });

        // Get Jesque client
        Client client = (Client) injector.getInstance(Client.class);
        LOG.info("Publish jobs");
        // Push jobs
        client.enqueue("EXP_QUEUE", new Job("ExampleJob", "hello", "job1"));
        client.enqueue("EXP_QUEUE", new Job("ExampleJob", "hello", "job2"));
    }
}

You should see,

// Job - 1
DEBUG c.s.commons.jesque.GuiceAwareWorker - Injecting dependencies into worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6b754699
DEBUG c.s.commons.jesque.GuiceAwareWorker - Delegating to run worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6b754699
INFO  c.s.c.jesque.example.ExampleWorker - Running worker=class com.strumsoft.commons.jesque.example.ExampleWorker, with arg1=hello, arg2=job1
INFO  c.s.c.jesque.example.ExampleService - Heya! I am not here to serve, just to show you how injection works.
INFO  c.s.c.jesque.example.EchoListener - onEvent ==>> queue=EXP_QUEUE, event=JOB_SUCCESS

// Job -2
DEBUG c.s.commons.jesque.GuiceAwareWorker - Injecting dependencies into worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6602e323
DEBUG c.s.commons.jesque.GuiceAwareWorker - Delegating to run worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6602e323
INFO  c.s.c.jesque.example.ExampleWorker - Running worker=class com.strumsoft.commons.jesque.example.ExampleWorker, with arg1=hello, arg2=job2
INFO  c.s.c.jesque.example.ExampleService - Heya! I am not here to serve, just to show you how injection works.
INFO  c.s.c.jesque.example.EchoListener - onEvent ==>> queue=EXP_QUEUE, event=JOB_SUCCESS

Works… eh? 🙂

You can grab the project source at github https://github.com/anismiles/jesque-guice Give it a try. I hope this will help some of you folks. Share your thoughts with me.

Happy hacking!

Written by Animesh

September 18, 2013 at 4:44 pm

Delayed Jobs with Jesque

with 4 comments

Jesque is an interesting project. It’s a Java port of Github’s Resque task queue library. Works on top of Redis. It is fast. It eats less. And doesn’t need a lot of maintenance, as long long your Redis is okay, you are okay! We have been happily using this in production for a while.

Problem

And then, we encountered a need for delayed jobs. We needed an ability to execute jobs in future with deterministic delay. We had 2 options:

1. Either, schedule these delayed jobs in Quartz – that was okay because we already had been using Quartz – and then, once Quartz jobs get fired, they publish a Task into Jesque, and let the workers handle the rest. (This was too contrived to implement, and would have become maintenance/reporting nightmare!)

2. We extend Jesque to support delayed jobs inherently.

Solution

We decided to go with option-2, and started exploring Redis datasets. Turned out, ZSET was all that we needed.

Jesque uses Redis LIST for job storage, workers keep polling the list, and LPOPs tasks from the LIST.

This is what we ended up doing:

When adding a delayed job,

1. Calculate future timestamp when the job should run,
2. Use that timestamp as SCORE to ZSET entry.

// Java
final long delay = 10; //sec
final long future = System.currentTimeMillis() + (delay * 1000); // future
jedis.zadd(QUEUE, future, jobInfo); 

// Redis
// ZADD <queue> <future> <job-information>

On the other hand, Workers’ poll logic was updated. For Delayed queues,

1. Check if there are any items with SCORE between -INF and now,

final long now = System.currentTimeMillis();
final Set<String> tasks = jedis.zrangeByScore(QUEUE, -1, now, 0, 1);

If tasks are non-empty, try to grab one to execute.

if (null != tasks && !tasks.isEmpty()) {
    String task = tasks.iterator().next();
    // try to acquire this task
    if (jedis.zrem(QUEUE, task) == 1) {
         return task; // Return
    }
}

This way, we ensure that no 2 workers would grab the same task to execute.

Also, an important point to note here is that – You don’t have to change your existing workers or redo new workers in any way. Just bind them to a Delayed Queue, and start publishing delayed tasks.

Example

// DelayedJobTest.java
package net.greghaines.jesque;

import static net.greghaines.jesque.utils.JesqueUtils.entry;
import static net.greghaines.jesque.utils.JesqueUtils.map;

import java.util.Arrays;

import net.greghaines.jesque.client.Client;
import net.greghaines.jesque.client.ClientPoolImpl;
import net.greghaines.jesque.worker.Worker;
import net.greghaines.jesque.worker.WorkerImpl;
import redis.clients.jedis.JedisPool;

public class DelayedJobTest {

    @SuppressWarnings("unchecked")
    public static void main(String[] args) throws InterruptedException {
        // Queue name
        final String QUEUE = "fooDelayed";
        
        // Config
        final Config config = new ConfigBuilder().withHost("localhost").withPort(6379).withDatabase(0).build();
        
        // Client
        final Client client = new ClientPoolImpl(config, new JedisPool("localhost"));
        long delay = 10; // seconds
        long future = System.currentTimeMillis() + (delay * 1000); // Future timestamp
        
        // Enqueue job
        client.delayedEnqueue(QUEUE, 
                new Job(
                        TestJob.class.getSimpleName(), 
                        new Object[] {"HELLO", "WORLD" } // arguments
                ), 
                future);
        // End
        client.end();

        // Worker
        final Worker worker = new WorkerImpl(config, Arrays.asList(QUEUE), map(entry(TestJob.class.getSimpleName(), TestJob.class)));
        final Thread workerThread = new Thread(worker);
        workerThread.start(); // start
    }
}

And, this is the TestJob,

// TestJob.java
package net.greghaines.jesque;

import java.util.Date;

public class TestJob implements Runnable {
    final String arg1;
    final String arg2;

    public TestJob(String arg1, String arg2) {
        this.arg1 = arg1;
        this.arg2 = arg2;
    }

    @Override
    public void run() {
        System.out.println("Job ran at=" + new Date() + ", with arg1=" + arg1 + " and arg2=" + arg2);
    }
}

We have using this solution in production for quite sometime, and this has been pretty stable.

BTW, you can grab the source here: https://github.com/anismiles/jesque and give it a try yourself.

Happy Hacking!

Written by Animesh

September 2, 2013 at 5:06 pm

Jetty Executable Webserver Archetype

with 2 comments

We create and deploy many rest based API apps on our servers. Now, instead of managing multitudes of webservers on different machines, we make our deployable archives wrapped with all the necessary dependencies in order to be self-executable. Benefits?

  • We just have to ensure that all our servers have the same Java environment, and no other dependencies.
  • No waste of time configuring, managing web (tomcat, jetty etc.) servers.
  • And, we only have to distribute a single executable.

Hassle free management. Won’t you agree?

To cut down on bootstrapping time, we have created a maven archetype to help us quickly create and start on new API projects. Our tech stack is Google Guice, JBoss Resteasy, Logback, and Embedded Jetty.

Github link: https://github.com/anismiles/jetty-webserver-archetype

Install

You can either build/install from source

git clone https://github.com/anismiles/jetty-webserver-archetype.git
mvn clean install

Or you can simply download the jar and install that.

mvn install:install-file \
-Dfile=jetty-webserver-archetype-1.0.jar \
-DgroupId=com.strumsoft \
-DartifactId=jetty-webserver-archetype \
-Dversion=1.0 \
-Dpackaging=jar \
-DgeneratePom=true

Usage

Let’s say you want to create a new project “hello-world” with group “com.hello.world”, you would run:

mvn archetype:generate \
-DgroupId=com.hello.world \
-DartifactId=hello-world \
-Dversion=1.0-SNAPSHOT \
-DarchetypeGroupId=com.strumsoft \
-DarchetypeVersion=1.0 \
-DarchetypeArtifactId=jetty-webserver-archetype

That’s it. You are ready to roll. You have set up a basic working API app. Now, you can run it in dev mode

mvn clean jetty:run

Or in production mode

mvn clean package
java –jar target/hello-world-1.0-SNAPSHOT-dist.war start &

To check, open http://localhost:8085/
To stop:

java –jar target/hello-world-1.0-SNAPSHOT-dist.war stop

Further, you can pass additional Jetty, Logback or your App’s properties

java \
-Djetty.configurationFile=<jetty-config> \
-Dapp.configurationFile=<app.properties> \
-Dlogback.configurationFile=<logback.xml> \
–jar target/hello-world-1.0-SNAPSHOT-dist.war start &

They will override default properties setup by the executable.
Ideally you might want to create an init.d file to start and stop you API app efficiently. Here is a prototype:

#!/bin/sh

# Executable war file
WAR_FILE=/var/apps/<my-war-file>.war

# Configuration files
APP_NAME=<app-name>

JETTY_CONFIG=/etc/${APP_NAME}-jetty.properties
APP_CONFIG=/etc/${APP_NAME}-app.properties
LOGBACK_CONFIG=/etc/${APP_NAME}-logback.properties

# Process
PIDFILE=/var/run/${APP_NAME}.pid

# Java arguments
JAVA_ARGS="-Djetty.configurationFile=${JETTY_CONFIG} -Dapp.configurationFile=${APP_CONFIG} -Dlogback.configurationFile=${LOGBACK_CONFIG}"

# Command
CMD=$1

if [[ -z ${CMD} ]]; then
  java ${JAVA_ARGS} -jar ${WAR_FILE} usage
  exit 1
fi

if [[ ${CMD} == 'start' ]]; then
    if [[ -f ${PIDFILE} ]]; then
        echo "Already running"
        exit 1
    fi

    if [[ -f ${WAR_FILE} ]]; then
        echo "Starting jetty: ${WAR_FILE}"
        java ${JAVA_ARGS} -jar ${WAR_FILE} start &
        PID=$!
        echo "$PID" > ${PIDFILE}
        echo "Started ${APP_NAME} with pid: ${PID}"
    fi

elif [[ ${CMD} == 'stop' ]]; then
    # Try gracefully first
    java ${JAVA_ARGS} -jar ${WAR_FILE} stop
    sleep 10
    if [[ -f ${PIDFILE} ]]; then
        PID=`cat ${PIDFILE}`
        test -z $PID || kill $PID
        rm ${PIDFILE}
        sleep 1
        echo "Forcibly Stopped ${WAR_FILE} with pid: ${PID}"
    fi

else # Just let the other cmds through...
    java ${JAVA_ARGS} -jar ${WAR_FILE} ${CMD}
fi

exit 0

That’s it! have fun. 🙂

Written by Animesh

January 8, 2013 at 1:48 pm

Securing, Versioning and Auditing REST (JAX-RS, Jersey) APIs

with 33 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

Now that your functionalities are working, you want a layer of security to authenticate/authorize your APIs. Though this is a bad approach towards security, but – I know – real life is a tough game and nothing happens they way they should be… and so be it. Additionally, you might want to control API versions (i.e. expose newer APIs only to newer clients) and audit API usage.

Well, I am going to propose a tangential way to implement these concerns. You won’t need to touch any of your business logic as such. Only few annotations (custom and otherwise) would need to be applied. That way, you won’t feel bad about missing these things when you started the project and your concerns will be taken care of in the most un-obtrusive way possible. Wonderful… eh?

First, you will need to create some sort of sign-in API, which will accept username/password (or oAuth or whatever you fancy) and generate some sort of session information which you will store in some database (Redis maybe!) and share its ID, say sessionId, with client. Then, with every subsequent request, Client will attach this sessionId in the request header which server will pick and look up for associated session information (permission, roles etc.) and based upon that server will authenticate and authorize the request.

Here is your Session bean.

package com.strumsoft.api;

import java.io.Serializable;
import java.util.Date;

/**
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 *
 */
public class Session implements Serializable {
	// 
	private static final long serialVersionUID = -7483170872690892182L;
	
	private String sessionId;   // id
	private String userId;      // user
	private boolean active;     // session active?
	private boolean secure;     // session secure?

	private Date createTime;    // session create time
	private Date lastAccessedTime;  // session last use time

	// getters/setters here
}

And this is your User bean. This must implement java.security.Principal.


package com.strumsoft.api;

import java.util.Set;

/**
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 *
 */
public class User implements java.security.Principal {
	// Role
	public enum Role {
		Editor, Visitor, Contributor
	};

	private String userId;          // id
	private String name;            // name
	private String emailAddress;    // email
	private Set<Role> roles;        // roles

	@Override
	public String getName() {
		return null;
	}

	// getters/setters here
}

Now, you need to implement javax.ws.rs.core.SecurityContext. This will be bound to incoming request and will decide whether to allow or deny it.

package com.strumsoft.api;

import java.security.Principal;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.SecurityContext;

/**
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 *
 */
public class MySecurityContext implements javax.ws.rs.core.SecurityContext {

	private final User user;
	private final Session session;

	public MySecurityContext(Session session, User user) {
		this.session = session;
		this.user = user;
	}

	@Override
	public String getAuthenticationScheme() {
		return SecurityContext.BASIC_AUTH;
	}

	@Override
	public Principal getUserPrincipal() {
		return user;
	}

	@Override
	public boolean isSecure() {
		return (null != session) ? session.isSecure() : false;
	}

	@Override
	public boolean isUserInRole(String role) {

		if (null == session || !session.isActive()) {
			// Forbidden
			Response denied = Response.status(Response.Status.FORBIDDEN).entity("Permission Denied").build();
			throw new WebApplicationException(denied);
		}

		try {
			// this user has this role?
			return user.getRoles().contains(User.Role.valueOf(role));
		} catch (Exception e) {
		}
		
		return false;
	}
}

Then, you need a ResourceFilter which will intercept the request, look for sessionId in the header and generate and attach our SecurityContext implementation to it. Notice, how our implementation only gets applied on Request but not on Response.

package com.strumsoft.api;

import javax.ws.rs.ext.Provider;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.flockthere.api.repository.SessionRepository;
import com.flockthere.api.repository.UserRepository;
import com.sun.jersey.spi.container.ContainerRequest;
import com.sun.jersey.spi.container.ContainerRequestFilter;
import com.sun.jersey.spi.container.ContainerResponseFilter;
import com.sun.jersey.spi.container.ResourceFilter;

/**
 * Filter all incoming requests, look for possible session information and use that
 * to create and load a SecurityContext to request. 
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 * 
 */
@Component   // let spring manage the lifecycle
@Provider    // register as jersey's provider
public class SecurityContextFilter implements ResourceFilter, ContainerRequestFilter {

	@Autowired
	private SessionRepository sessionRepository;  // DAO to access Session

	@Autowired
	private UserRepository userRepository;  // DAO to access User

	
	@Override
	public ContainerRequest filter(ContainerRequest request) {
		// Get session id from request header
		final String sessionId = request.getHeaderValue("session-id");

		User user = null;
		Session session = null;

		if (sessionId != null && sessionId.length() > 0) {
			// Load session object from repository
			session = sessionRepository.findOne(sessionId);
			
			// Load associated user from session
			if (null != session) {
				user = userRepository.findOne(session.getUserId());
			}
		}

		// Set security context
		request.setSecurityContext(new MySecurityContext(session, user));
		return request;
	}

	@Override
	public ContainerRequestFilter getRequestFilter() {
		return this;
	}

	@Override
	public ContainerResponseFilter getResponseFilter() {
		return null;
	}
}

Okay, the hard part is over. All we need now is a way to fire our SecurityContextFilter. For this, we will create a ResourceFilterFactory. During application startup, this factory will create a List of filters for all AbstractMethods of each of our Resources. We are going to extend RolesAllowedResourceFilterFactory which will generate all Role based ResourceFilters for us. And then, we will add our SecurityContextFilter on the top of the list with VersionFilter and AuditFilter in the bottom. That way, SecurityContextFilter will executed first because you need to make auth decisions early. VersionFilter will be next. And Audit in the bottom. You want to audit when everything else has been done. No?

package com.strumsoft.api;

import java.util.ArrayList;
import java.util.List;

import javax.ws.rs.ext.Provider;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.flockthere.api.AllowAllVersions;
import com.flockthere.api.audit.Audit;
import com.flockthere.api.resource.interceptor.AuditingFilter;
import com.flockthere.api.resource.interceptor.SecurityContextFilter;
import com.flockthere.api.resource.interceptor.VersionFilter;
import com.sun.jersey.api.container.filter.RolesAllowedResourceFilterFactory;
import com.sun.jersey.api.model.AbstractMethod;
import com.sun.jersey.spi.container.ResourceFilter;

/**
 * FilterFactory to create List of request/response filters to be applied on a particular
 * AbstractMethod of a resource.
 * 
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 * 
 */
@Component // let spring manage the lifecycle
@Provider  // register as jersey's provider
public class ResourceFilterFactory extends RolesAllowedResourceFilterFactory {

	@Autowired
	private SecurityContextFilter securityContextFilter;

	// Similar to SecurityContextFilter to check incoming requests for API Version information and
	// act accordingly
	@Autowired
	private VersionFilter versionFilter;

	// Similar to SecurityContextFilter to audit incoming requests
	@Autowired
	private AuditingFilter auditingFilter;

	@Override
	public List<ResourceFilter> create(AbstractMethod am) {
		// get filters from RolesAllowedResourceFilterFactory Factory!
		List<ResourceFilter> rolesFilters = super.create(am);
		if (null == rolesFilters) {
			rolesFilters = new ArrayList<ResourceFilter>();
		}

		// Convert into mutable List, so as to add more filters that we need
		// (RolesAllowedResourceFilterFactory generates immutable list of filters)
		List<ResourceFilter> filters = new ArrayList<ResourceFilter>(rolesFilters);

		// Load SecurityContext first (this will load security context onto request)
		filters.add(0, securityContextFilter);

		// Version Control?
		filters.add(versionFilter);

		// If this abstract method is annotated with @Audit, we will apply AuditFilter to audit
		// this request.
		if (am.isAnnotationPresent(Audit.class)) {
			filters.add(auditingFilter);
		}

		return filters;
	}
}

VersionFilter will help control Client’s access to API methods based upon client’s version. Implementation would be similar to SecurityContextFilter. You will need to annotate API methods with @Version(“>1.3”). VersionFilter will read this value (“>1.3”), check request headers for API-Version keys and then decide whether to allow or reject the request. Similarly, AuditFilter will log all such annotated (@Audit(“audit-note”)) API methods. I will not discuss their actual implementations. You can very easily write them based upon your requirement or can remove them altogether if not needed.

Here is how these annotations will look like.

// @Version
@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.METHOD })
public @interface Version {
	String value();
}

// @Audit
@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.METHOD })
public @interface Audit {
	String value();
}

And now, you will need to update you web.xml to include ResourceFilterFactory

...
	<servlet>
		<servlet-name>REST</servlet-name>
		<servlet-class>
			com.sun.jersey.spi.spring.container.servlet.SpringServlet
		</servlet-class>
		...		
		<!-- Our resource filter for tangential concerns (security, logging, version etc.) -->
		<init-param>
			<param-name>com.sun.jersey.spi.container.ResourceFilters</param-name>
			<param-value>com.strumsoft.api.ResourceFilterFactory</param-value>
		</init-param>
		...		
	</servlet>
...

That’s it. All configuration is done. Now, you just need to annotate you Resources. Let’s see how.

Say, you have a BookResource which exposes APIs to create/edit/list books. And you want
1. “Editor/Contributor” to be able to create a book, (@RolesAllowed({ “Editor”, “Contributor” }))
2. Only “Editor” to be able to edit a book, (@RolesAllowed({ “Editor” }))
3. While, anyone can see all the books. (@PermitAll)

Also, let’s say editing a book was released in API 1.3, so any client still using older APIs should not be able to update (@Version(“>1.3”)) an existing book. (Assuming you implemented VersionFilter properly)

Additionally, create or edit book should be audited with respective notes “create-book” and “edit-book” given your AuditFilter is in place.

package com.strumsoft.api;

import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.SecurityContext;

import com.flockthere.api.audit.Audit;

/**
 * Book Resource
 * 
 * @Path("/book/")
 * @author "Animesh Kumar <animesh@strumsoft.com>"
 * 
 */
@Produces({ MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON })
public interface BookResource {

	// Only Editor and Contributor can create a book entry
	@javax.annotation.security.RolesAllowed({ "Editor", "Contributor" })
	@Audit("create-book")  // Audit
	@POST
	public Book createBook(@Context SecurityContext sc, @FormParam("title") String title,
			@FormParam("author") String author, @FormParam("publisher") String publisher,
			@FormParam("isbn") String isbn, @FormParam("edition") String edition);

	// Only Editor can edit an existing book entry
	@javax.annotation.security.RolesAllowed({ "Editor" })
	@Audit("edit-book")   // Audit
	@Version(">1.3")  // Available only on API version 1.3 onwards
	@PUT
	public Book editBook(@Context SecurityContext sc, @FormParam("title") String title,
			@FormParam("author") String author, @FormParam("publisher") String publisher,
			@FormParam("isbn") String isbn, @FormParam("edition") String edition);
	
	// Anyone can see these books
	@javax.annotation.security.PermitAll
	@GET
	public Book listBooks();

}

And at last, you will need to add jsr250-api to your project dependencies which defines javax.annotation.security.* annotations.

	<!-- securty tags: javax.annotation.security.* (@RolesAllowed, @PermitAll, @DenyAll etc.) -->
	<dependency>
		<groupId>javax.annotation</groupId>
		<artifactId>jsr250-api</artifactId>
		<version>1.0</version>
	</dependency>

Chill. 🙂

Written by Animesh

March 2, 2012 at 7:19 pm

Posted in Technology

Tagged with , , , , ,

Fun with Singleton (Python, Javascript, Java)

leave a comment »

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

They say that Singletons, like global variables, are evil. They hide dependencies; are harder to test and even harder to extend. Singletons are lies, and it’s best to keep away from them. But, there are scenarios where you need them. For example, when you want a shared resource like printer spooler or file manager or log manager, you want a single object to handle requests from all the various parts of your application.

In this blog, I am going to explore various ways to make Singletons in languages like Python, Java and Javascript so as to keep it simple, elegant and usable. Let’s talk about Python first. I love it, and it’s a really really wonderful language, and in here, there are n different ways to solve a problem. Singletons are no exception. The most natural way to do it is to create a decorator.

class Singleton(object):
	def __init__(self, klass):
		self.klass = klass   # class which is being decorated
		self.instance = None  # instance of that class
	def __call__(self, *args, **kwargs):
		if self.instance is None:
			# new instance is created and stored for future use
			self.instance = self.klass(*args, **kwargs)
		return self.instance

Now, let’s say you have a Resource class. To make it singleton, you just need to decorate it with ‘@Singleton‘, and you are done.

@Singleton
class Resource(object):
	def __init__(self, klass):
		self.name = None

Cool…eh? There are other – nerdy – ways too. Python uses an internal dictionary ‘__dict__’ variable to keep track of an Object’s properties and methods. So, if you can share ‘__dict__‘ across multiple instances of a Class, you can share the state between them. And isn’t that Singleton? Yes, that is. You might have many many instances, but all of them behave exactly the same.

class Singleton(object):
	_shared = {}
	def __init__(self):
		self.__dict__ = Singleton._shared
class Resource(Singleton):
	def __init__(self, klass):
		self.name = None

Since ‘self.__dict__‘ now refers to ‘_shared‘ dictionary, all instances of Resource would use the same dictionary and hence they will all have the same behavior. Geeky? Let me show you an even geekier way to do it.

In Python, when you instantiate a class, the interpreter calls ‘__new__‘ method – a class method which returns an instance of the class – and then ‘__init__‘ method – constructor of the class – is called which initializes the object. So, you can hack into ‘__new__‘ and return the single instance whenever it is being called.

class Singleton(object):
	_instance = None
	def __new__(cls, *args, **kwargs):
	# cls is the Class and the rest are constructor arguments
		if cls._instance is None:
			# create an instance and store it
			cls._instance = Object.__new__(cls, *args, **kwargs)
		return cls._instance
class Resource(Singleton):
	def __init__(self, klass):
		self.name = None

Aweomse! Isn’t it? There are other ways that deal with ‘__metaclass__‘ etc. but let’s save them for another day. Let’s use it now:

# get resource r1
r1 = Resource();
# get resource r2  (since Resource is singleton, r1 == r2)
r2 = Resource();
# to verify, let's set 'name' onto r1
r1.name = "Animesh Kumar"
print r1.name
# and the same 'name' appears in r2 as well!
print r2.name

Let’s now see how do we do this in Javascript. For the most simple form, just define an Object Literal, and you are done.

var resource = {
	getName : function() {
		return this.name;
	},
	setName: function(name){
		this.name = name;
	}
}

Easy. You have an object which you can now share across your application modules and it just works. For more complex scenarios, like private variables and all, you might have to resort to something like this:

// self-executable wrapper function
var Resource = (function(){
	// Resouce class which is to made 'singleton'
	function _Resource() {
    	var name; // private variable
    	this.getName = function() {	// getter
    		return this.name;
    	};
		this.setName= function(name){ // setter
			this.name = name;
		};
		// do more stuffs
    }
    // instance holder
    var instance = new _Resource();</p>
    // return an object with 'getInstance' method
    return = {
        getInstance: function(){
            return instance;
        }
   };
})();

_Resource (line-04) is your function of interest, and you want to make it singleton. So, you create another function ‘Resource‘ which wraps over _Resource and returns an object with method ‘getInstance‘ which would return the same instance of _Resource every time it will be called.

Let’s try to use it now:

// get resource r1
r1 = Resource.getInstance();
// get resource r2  (since Resource is singleton, r1 == r2)
r2 = Resource.getInstance();
// to verify, let's set 'name' onto r1
r1.setName("Animesh Kumar");
console.log(r1.getName());
// and the same 'name' appears in r2 as well!
console.log(r2.getName());

So it was easy. No? Great.

Now, Java. I know every one of you would already know it. I would write about it anyway, just for the sake of completeness. In Java, you create a private static instance of the class, and use that instance wherever necessary.

public class Resource {
	// static instance (Note: Resource instantiation is done here, not in getInstance)
	private static Resource instance = new Resource();
	// property
	private String name;
	// private constructor to disable 'new'
	private Resource() {
	}
	// public staic method to get an instance of this class
	public static Resource getInstance() {
		return instance;
	}
	// getter
	public String getName() {
		return name;
	}
	// setter
	public void setName(String name) {
		this.name = name;
	}
}

Now, let’s use it.

	public static void main(String[] args) {
		// get resource r1
		Resource r1 = Resource.getInstance();
		// get resource r2  (since Resource is singleton, r1 == r2)
		Resource r2 = Resource.getInstance();
		// to verify, let's set 'name' onto r1
		r1.setName("Animesh Kumar");
		System.out.println(r1.getName());
		// and the same 'name' appears in r2 as well!
		System.out.println(r2.getName());
	}

Loud and clear. And this also stops you from instantiating ‘Resource‘ with ‘new‘ operator. Try this:

	Resource r1 = new Resource();  // java.lang.Error: Unresolved compilation problem:

Your code will not compile. I know you know why. I will write anyways: because the constructor is private! So, there is no way to get an instance of Resource class but through ‘getInstance’ method which ensures single instance of the class. Also, If you noticed: I have instantiated ‘instance’ during declaration itself, not in ‘getInstance’ method. This way, the object gets created at the time of class loading, and you save yourself from a lot of issues that creeps in because of Java Just in Time.

Written by Animesh

May 27, 2011 at 3:59 pm

shutdown hooks in java

with one comment

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

You know what? You can add your custom hooks to JVM to be called when JVM shuts down. Interesting… no?

What happens is very simple: you create a Thread and write you logic in there, and then register this Thread to the Runtime instance. Here is a sample:

public class JVMShutdownTest {

	public static void main(String[] args) {

		// Add a shutdown hook
		Runtime.getRuntime().addShutdownHook(new Thread(){
			public void run(){
				System.out.println ("closing down the shop...");
			}
		});

		// Exit now
		System.exit(0);
	}
}

output will be: closing down the shop…

This attached Thread is initialized but not yet started. When JVM starts to shut down, it starts all registered hooks in an uncontrollable (that is, there is no way to enforce your will) order, and all of them run concurrently.

One thing you must take in consideration is that these hooks get executed at a very delicate time and so you must keep them light, thread-safe and independent of heavy dependencies. Thumb rule: hooks must finish quick.

Read more >>

Written by Animesh

March 28, 2011 at 4:41 pm

Posted in Technology

Tagged with ,

Postgres Enum with Hibernate

with 8 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

In a running project, we had to map Postgres Enum onto Java Enum using Hibernate. It turned out to be more tricky than we had assumed initially. So, I thought a little tutorial here might save your time if you are stuck in the similar situation.

I am taking an example of Person-Gender mapping where Gender is of enum type.

  1. First, create proper Enums and Tables in Postgres database. Here is the script.
    create type genderType as enum (
    	'MALE', 'FEMALE'
    );
    
    create table person (
    	personid serial NOT NULL,
    	gender genderType not null
    );
    
  2. Then, you need to define you Enum class.
    package com.impetus.ilabs.entity;
    
    public enum GenderType {
        MALE, FEMALE;
    }
    
  3. Now, define your Entity class like this,
    package com.impetus.ilabs.entity;
    
    import java.io.Serializable;
    
    public class Person implements Serializable {
    
    	private Integer personId;
    	
    	private GenderType gender;
    
    	public Integer getPersonId() {
    		return personId;
    	}
    
    	public void setPersonId(Integer personId) {
    		this.personId = personId;
    	}
    
    	public GenderType getGender() {
    		return gender;
    	}
    
    	public void setGender(GenderType gender) {
    		this.gender = gender;
    	}
    
    	@Override
    	public String toString() {
    		StringBuilder builder = new StringBuilder();
    		builder.append("Person [gender=");
    		builder.append(gender);
    		builder.append(", personId=");
    		builder.append(personId);
    		builder.append("]");
    		return builder.toString();
    	}
    }
    
  4. This is the magical part. You need to implement a custom UserType that can understand Enum objects. Here is an implementation of PGEnumUserType.
    package com.impetus.ilabs;
    
    import java.io.Serializable;
    import java.sql.PreparedStatement;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import java.sql.Types;
    import java.util.Properties;
    
    import org.hibernate.HibernateException;
    import org.hibernate.usertype.EnhancedUserType;
    import org.hibernate.usertype.ParameterizedType;
    import org.postgresql.util.PGobject;
    
    // This implementation works only with Postgres
    public class PGEnumUserType implements EnhancedUserType, ParameterizedType {
    	// Enum  class under observation
    	private Class<Enum> enumClass;
    
    	public void setParameterValues(Properties parameters) {
    		String enumClassName = parameters.getProperty("enumClassName");
    		try {
    			enumClass = (Class<Enum>) Class.forName(enumClassName);
    		} catch (ClassNotFoundException cnfe) {
    			throw new HibernateException("Enum class not found", cnfe);
    		}
    	}
    
    	public Object assemble(Serializable cached, Object owner)
    			throws HibernateException {
    		return cached;
    	}
    
    	public Object deepCopy(Object value) throws HibernateException {
    		return value;
    	}
    
    	public Serializable disassemble(Object value) throws HibernateException {
    		return (Enum) value;
    	}
    
    	public boolean equals(Object x, Object y) throws HibernateException {
    		return x == y;
    	}
    
    	public int hashCode(Object x) throws HibernateException {
    		return x.hashCode();
    	}
    
    	public boolean isMutable() {
    		return false;
    	}
    
    	public Object nullSafeGet(ResultSet rs, String[] names, Object owner)
    			throws HibernateException, SQLException {
    		Object object = rs.getObject(names[0]);
    		if (rs.wasNull()) {
    			return null;
    		}
    
    		// Notice how Object is mapped to PGobject. This makes this implementation Postgres specific
    		if (object instanceof PGobject) {
    			PGobject pg = (PGobject) object;
    			return Enum.valueOf(enumClass, pg.getValue());
    		}
    		return null;
    	}
    
    	public void nullSafeSet(PreparedStatement st, Object value, int index)
    			throws HibernateException, SQLException {
    		if (value == null) {
    			st.setNull(index, Types.VARCHAR); 
    			// UPDATE: To support NULL insertion, change to: st.setNull(index, 1111);
    		} else {
    			// Notice 1111 which java.sql.Type for Postgres Enum
    			st.setObject(index, ((Enum)value), 1111);
    		}
    	}
    
    	public Object replace(Object original, Object target, Object owner)
    			throws HibernateException {
    		return original;
    	}
    
    	public Class returnedClass() {
    		return enumClass;
    	}
    
    	public int[] sqlTypes() {
    		return new int[] { Types.VARCHAR }; 
    		// UPDATE: To support NULL insertion, change to: return new int[] { 1111 };
    	}
    
    	public Object fromXMLString(String xmlValue) {
    		return Enum.valueOf(enumClass, xmlValue);
    	}
    
    	public String objectToSQLString(Object value) {
    		return '\'' + ((Enum) value).name() + '\'';
    	}
    
    	public String toXMLString(Object value) {
    		return ((Enum) value).name();
    	}
    }
    
  5. Okay, so the tricky part is over. Now, you need to write Hibernate Mapping xml for Person class.
    <?xml version="1.0"?>
    <!DOCTYPE hibernate-mapping PUBLIC
            "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
            "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
    
    <hibernate-mapping>
    	<class name="com.impetus.ilabs.entity.Person" table="person">
    		<id name="personId" type="java.lang.Integer"
    			column="personid">
    			<generator class="native"></generator>
    		</id>
    
    		<property name="gender">
    			<type name="com.impetus.ilabs.PGEnumUserType">
    				<param name="enumClassName">com.impetus.ilabs.entity.GenderType</param>
    			</type>
    		</property>
    	</class>
    </hibernate-mapping>
    
  6. Done! I am including a sample usage class, but you don’t worry about this part, you are good with whatever DAO implementation you might have.
    package com.impetus.ilabs;
    
    import org.hibernate.Session;
    import org.hibernate.SessionFactory;
    import org.hibernate.cfg.Configuration;
    
    import com.impetus.ilabs.entity.GenderType;
    import com.impetus.ilabs.entity.Person;
    
    public class App {
    	
    	private static final SessionFactory sessionFactory = buildSessionFactory();
    
    	private static SessionFactory buildSessionFactory() {
    		try {
    			return new Configuration().configure().buildSessionFactory();
    		} catch (Throwable ex) {
    			throw new ExceptionInInitializerError(ex);
    		}
    	}
    	
    	private static void save (Person person) {
    		Session session = sessionFactory.openSession();
    		session.beginTransaction();
    			
    		// create object
    		session.save(person);
    		session.getTransaction().commit();
    	}
    
    	private static void load (int id) {
    		Session session = sessionFactory.openSession();
    		session.beginTransaction();
    			
    		System.out.println (session.load(Person.class, id));
    		session.getTransaction().commit();
    	}
    
    	public static void main(String[] args) {
    		// create object
    		Person person = new Person();
    		person.setGender(GenderType.MALE);
    		save(person);
    		
    		// load object
    		load(person.getPersonId());
    	}
    }
    

Enjoy! By the way, the whole thing that is described above is also available as maven project to download.

Written by Animesh

August 4, 2010 at 3:48 pm

Posted in Technology

Tagged with , , , ,

JUnit Primer

leave a comment »

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

I recently gave a presentation on JUnit and thought it might be useful to other people as well. So, I’m sharing the PPT here.

Written by Animesh

August 2, 2010 at 2:20 pm

Posted in Technology

Tagged with , ,

Discovering Java Annotations

with 15 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

DZone link of this artilce: http://www.dzone.com/links/r/discovering_java_annotations.html

Annotations these days are proving to be widely popular. Many Annotation based frameworks (Spring, JPA, Tapestry to name few) are seeing the light of the day, and even small scale projects are using Annotation based meta-programming for better separation of concern. Annotations are wonderful and it’s sure to stay here.

However, in order to use this, we must be able to locate classes annotated with Annotations that concerns us. How do you do it?

First, you find resources where we can look for annotated classes. You can look into “java.class.path” system property (also known as CLASSPATH), or use Classloader or ServletContexts to have a list of Resources. Now, you scan through each resource and check for Annotations that concern you.

The easiest way to scan through a resource is to load it through a Classloader and use Java Reflection Api to look for a specified annotation. However, this approach will only help you to find Annotations that are visible at runtime, and loading each resource into memory will unnecessarily consume Java Memory. Otherwise, you can use ASM or Javassist byte processing libraries. These libraries process class bytes and see even runtime annotations. And since they don’t load your resources into Memory, they have a low footprint.

Well, to help you here, we have written a small library Annovention using Javassist. The idea behind Annovention is to help you quickly locate annotated classes, fields or methods. Once you have the pertinent classes you can run your domain logic or do whatever fancies you.

Download Annovention here: http://code.google.com/p/annovention/

How it works?

Annovention works on subscribe-and-listen pattern.

  1. You create annotation discovery listeners.
    1. Annovetion supports Class, Field and Method level annotation discovery and each of these listeners must implement ClassAnnotationDiscoveryListener, FieldAnnotationDiscoveryListener or MethodAnnotationDiscoveryListener interfaces respectively.
    2. Each listener has a method supportedAnnotations() that returns an Array of Annotation names. For example, to listen to @Entity and @EntityListeners annotations, the array will be:
      new String[] {Entity.class.getName(), EntityListeners.class.getName()}
      
    3. Each listener has a method discovered() that is called each time a relevant annotation is located with Class-name, Field-name, Method-name and discovered Annotation-name.
    4. Please note that your listeners are receiving only names of classes, fields and methods. You must use Java Reflection to instantiate them. Remember Class.forName()?
  2. Now, there is a Discoverer class that does all the discoveries. It needs Resources and Listeners. Annovention comes with an implementation of ClasspathDiscoverer. This uses “java.class.path” system property and builds an array of resources to scan. You need to register your Listeners to Discoverer class in order to get notified.

Sample Use

public class SampleAnnotationDiscoverer {

	public static void main (String args []) {
		// Get a classpath discoverer instance
		Discoverer discoverer = new ClasspathDiscoverer();

		// Register class annotation listener
		discoverer.addAnnotationListener(new MyClassAnnotationListener());
		// Register field annotation listener
		discoverer.addAnnotationListener(new MyFieldAnnotationListener());
		// Register method annotation listener
		discoverer.addAnnotationListener(new MyMethodAnnotationListener());

		// Fire it
		discoverer.discover();
	}

	/** Dummy ClassAnnotation listener */
	static class MyClassAnnotationListener implements ClassAnnotationDiscoveryListener {
		private static Log log =
			LogFactory.getLog(MyClassAnnotationListener.class);

		@Override
		public void discovered(String clazz, String annotation) {
			log.info("Discovered Class(" + clazz + ") " +
					"with Annotation(" + annotation + ")");
		}

		@Override
		public String[] supportedAnnotations() {
			// Listens for @Entity and @EntityListeners annotations.
			return new String[] {
					Entity.class.getName(),
					EntityListeners.class.getName()};
		}
	}

	/** Dummy FieldAnnotation listener */
	static class MyFieldAnnotationListener implements FieldAnnotationDiscoveryListener {
		private static Log log =
			LogFactory.getLog(MyFieldAnnotationListener.class);

		@Override
		public void discovered(String clazz, String field, String annotation) {
			log.info("Discovered Field(" + clazz + "." + field + ") " +
					"with Annotation(" + annotation + ")");
		}

		@Override
		public String[] supportedAnnotations() {
			// Listens for @Id and @Column annotations.
			return new String[] {
					Id.class.getName(),
					Column.class.getName()};
		}
	}

	/** Dummy FieldAnnotation listener */
	static class MyMethodAnnotationListener implements MethodAnnotationDiscoveryListener {

		private static Log log =
			LogFactory.getLog(MyMethodAnnotationListener.class);

		@Override
		public void discovered(String clazz, String method, String annotation) {
			log.info("Discovered Method(" + clazz + "." + method + ") " +
					"with Annotation(" + annotation + ")");
		}

		@Override
		public String[] supportedAnnotations() {
			// Listens for @PrePersist, @PreRemove and @PostPersist annotations.
			return new String[] {
					PrePersist.class.getName(),
					PostPersist.class.getName(),
					PreRemove.class.getName()};
		}
	}
}

How to extend?

You just have to play with Discoverer and Filter classes. The general flow is:

  1. Discoverer invokes findResources() method which returns an array of URLs and then
  2. It iterates through each URL and apply filter, and
  3. Then scan the resource for annotations and
  4. If a valid annotation is found, corresponding listeners are intimated.

You can write your own Filter implementation or use FilterImpl class that comes with Annovention. Then, extend Discoverer class and implement findResources() method in whatever way you see fit. Then just invoke discover() method. Easy?

Written by Animesh

July 26, 2010 at 6:08 pm

Posted in Technology

Tagged with , ,

Kundera: knight in the shining armor!

with 37 comments

[tweetmeme source=”anismiles” only_single=false http://www.URL.com%5D

The idea behind Kundera is to make working with Cassandra drop-dead simple, and fun. Kundera does not reinvent the wheel by making another client library; rather it leverages the existing libraries, and builds – on top of them – a wrap-around API to developers do away with the unnecessary boiler plate codes, and program  a neater, cleaner code that reduces code-complexity and improves quality. And above all, improves productivity.

Download Kundera here: http://code.google.com/p/kundera/

Note: Kundera is now JPA 1.0 compatible, and there are some ensuing changes. You should read about it here: https://anismiles.wordpress.com/2010/07/14/kundera-now-jpa-1-0-compatible/

Objectives:

  • To completely remove unnecessary details, such as Column lists, SuperColumn lists, byte arrays, Data encoding etc.
  • To be able to work directly with Domain models just with the help of annotations
  • To eliminate “code plumbing”, so as to keep the flow of data processing clear and obvious
  • To completely separate out Cassandra and its obvious concerns from application-level logics for robust application development
  • To include the latest Cassandra developments without breaking anything, anywhere in the business layer

Cassandra Data Models

At the very basic level, Cassandra has Column and SuperColumn to hold your data. Column is a tuple with a name, value and a timestamp; while SuperColumn is Column of Columns. Columns are stored in a ColumnFamily, and SuperColumns in SuperColumnFamily. The most important thing to note is that Cassandra is not your old relational database, it is a flat system. No joins, No foreign keys, nothing. Everything you store here is 100% de-normalized.

Read more details here: https://anismiles.wordpress.com/2010/05/18/cassandra-data-model/

Using Kundera

Kundera defines a range of annotations to describe your Entity objects. Kundera is now JPA1.0 compatible. It builds a range of various Annotations, on top of JPA annotations, to suit its needs. Here are the basic rules:

General Rules

  • Entity classes must have a default no-argument constructor.
  • Entity classes must be annotated with @CassandraEntity @Entity (@CassandraEntity annotation is dropped in favor of JPA @Entity)
  • Entity classes for ColumnFamily must be annotated with @ColumnFamily(“column-family-name”)
  • Entity classes for SuperColumnFamily must be annotated with @SuperColumnFamily(“super-column-family-name”)
  • Each entity must have a field annotation with @Id
    • @Id field must of String type. (Since you can define sorting strategies in Cassandra’s storage-conf file, keeping @Id of String type makes life simpler, you will see later)
    • There must be 1 and only 1 @Id per entity.

Note: Kundera works only at property level for now, so all method level annotations are ignored. Idea: keep life simple. 🙂

ColumnFamily Rules

  1. You must define the name of the column family in @ColumnFamily, like @ColumnFamily (“Authors”) Kundera will link this entity class with “Authors” column family.
  2. Entities annotated with @ColumnFamily are scanned for properties for @Colum annotations.
  3. Each such field will qualify to become a Cassandra Column with
    1. Name: name of the property.
    2. Value: value of the property
  4. By default the name of the column will be the name of the property. However, you fancy changing the name, you can override it like, @Column (name=”fancy-name”)
    @Column (name="email")          // override column-name
    String emailAddress;
    
  5. Properties of type Integer, String, Long and Date are inherently supported, rest all will be serialized before they get saved, and de-serialized while getting read. Serialization has some inherent limitations; that is why Kundera discourages you to use custom objects as Cassandra Column properties. However, you are free to do as you want. Just read the serialization tweaks before insanity reins over you, 😉
  6. Kundera also supports Collection and Map properties. However there are few things you must take care of:
    • You must initialize any Collection or Map properties, like
      List<String> list = new ArrayList<String>();
      Set<String> set = new HashSet<String>();
      Map<String, String> map = new HashMap<String, String>();
      
    • Type parameters follow the same rule, described in #5.
    • If you don’t explicitly define the type parameter, elements will be serialized/de-serialized before saving and retrieving.
    • There is no guarantee that the Collection element order will be maintained.
    • Collection and Map both will create as many columns as the number of elements it has.
    • Collection will break into Columns  like,
      1. Name~0: Element at index 0
      2. Name~1: Element at index 1 and so on.

      Name follows rule #4.

    • Map will break into Columns like,
      1. Name~key1: Element at key1
      2. Name~key2: Element at key2 and so on.
    • Again, name follows rule #4.

SuperColumnFamily Rules

  1. You must define the name of the super column family in @SuperColumnFamily, like @SuperColumnFamily (“Posts”) Kundera will link this entity class with “Posts” column family.
  2. Entities annotated with @SuperColumnFamily are scanned for properties for 2 annotations:
    1. @Column and
    2. @SuperColumn
  3. Only properties annotated with both annotations are picked up, and each such property qualifies to become a Column and fall under SuperColumn.
  4. You can define the name of the column like you did for ColumnFamily.
  5. However, you must define the name of the SuperColumn a particular Column must fall under like, @SuperColumn(column = “super-column-name”)
    @Column
    @SuperColumn(column = "post")  // column 'title' will fall under super-column 'post'
    String title;
    
  6. Rest of the things are same as above.

Up and running in 5 minutes

Let’s learn by example. We will create a simple Blog application. We will have Posts, Tags and Authors.

Cassandra data model for “Authors” might be like,

ColumnFamily: Authors = {
    “Eric Long”:{		// row 1
        “email”:{
            name:“email”,
            value:“eric (at) long.com”
        },
        “country”:{
            name:“country”,
            value:“United Kingdom”
        },
        “registeredSince”:{
            name:“registeredSince”,
            value:“01/01/2002”
        }
    },
    ...
}

And data model for “Posts” might be like,

SuperColumnFamily: Posts = {
	“cats-are-funny-animals”:{		// row 1
		“post” :{		// super-column
			“title”:{
				“Cats are funny animals”
			},
			“body”:{
				“Bla bla bla… long story…”
			}
			“author”:{
				“Ronald Mathies”
			}
			“created”:{
				“01/02/2010"
			}
		},
		“tags” :{
			“0”:{
				“cats”
			}
			“1”:{
				“animals”
			}
		}
	},
	// row 2
}

Create a new Cassandra Keyspace: “Blog”

<Keyspace Name="Blog">
<!—family definitions-->

<!-- Necessary for Cassandra -->
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>

Create 2 column families: SuperColumnFamily for “Posts” and ColumnFamily for “Authors”

<Keyspace Name="Blog">
<!—family definitions-->
<ColumnFamily CompareWith="UTF8Type" Name="Authors"/>
<ColumnFamily ColumnType="Super" CompareWith="UTF8Type" CompareSubcolumnsWith="UTF8Type" Name="Posts"/>

<!-- Necessary for Cassandra -->
<ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
<ReplicationFactor>1</ReplicationFactor>
<EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
</Keyspace>

Create entity classes

Author.java

@Entity			// makes it an entity class
@ColumnFamily ("Authors")	// assign ColumnFamily type and name
public class Author {

    @Id						// row identifier
    String username;

    @Column (name="email")	// override column-name
    String emailAddress;

    @Column
    String country;

    @Column (name="registeredSince")
    Date registered;

    String name;

    public Author () {		// must have a default constructor
    }

    ... // getters/setters etc.
}

Post.java

@Entity					// makes it an entity class
@SuperColumnFamily("Posts")			// assign column-family type and name
public class Post {

	@Id								// row identifier
	String permalink;

	@Column
	@SuperColumn(column = "post")	// column 'title' will be stored under super-column 'post'
	String title;

	@Column
	@SuperColumn(column = "post")
	String body;

	@Column
	@SuperColumn(column = "post")
	String author;

	@Column
	@SuperColumn(column = "post")
	Date created;

	@Column
	@SuperColumn(column = "tags")	// column 'tag' will be stored under super-column 'tags'
	List<String> tags = new ArrayList<String>();

	public Post () {		// must have a default constructor
	}

       ... // getters/setters etc.
}

Note the annotations, match them against the rules described above. Please see how “tags” property has been initialized. This becomes very important because Kundera uses Java Reflection to read and populate the entity classes. Anyways, once we have entity classes in place…

Instantiate EnityManager

Kundera now works as a JPA provider, and here is how you can instantiate EntityManager. https://anismiles.wordpress.com/2010/07/14/kundera-now-jpa-1-0-compatible/#entity-manager

EntityManager manager = new EntityManagerImpl();
manager.setClient(new PelopsClient());
manager.getClient().setKeySpace("Blog");

And that’s about it. You are ready to rock-and-roll like a football. Sorry, I just got swayed with FIFA fever. 😉

Supported Operations

Kundera supports JPA EntityManager based operations, along with JPA queries. Read more here: https://anismiles.wordpress.com/2010/07/14/kundera-now-jpa-1-0-compatible/#entity-operations


Save entities

Post post = ... // new post object
try {
manager.save(post);
} catch (IllegalEntityException e) { e.printStackTrace(); }
catch (EntityNotFoundException e) { e.printStackTrace(); }

If the entity is already saved in Cassandra database, it will be updated, else a new entity will be saved.
Load entity

try {
Post post = manager.load(Post.class, key); // key is the identifier, for our case, "permalink"
} catch (IllegalEntityException e) { e.printStackTrace(); }
catch (EntityNotFoundException e) { e.printStackTrace(); }

Load multiple entities

try {
List posts = manager.load(Post.class, key1, key2, key3...); // key is the identifier, "permalink"
} catch (IllegalEntityException e) { e.printStackTrace(); }
catch (EntityNotFoundException e) { e.printStackTrace(); }

Delete entity

try {
manager.delete(Post.class, key); // key is the identifier, "permalink"
} catch (IllegalEntityException e) { e.printStackTrace(); }
catch (EntityNotFoundException e) { e.printStackTrace(); }


Wow! Was it fun? Was it easy? I’m sure it was. Keep an eye on Kundera, we will be rolling out sooner-than-you-imagine more features like,

  1. Transaction support
  2. More fine-grained methods for better control
  3. Lazy-Loading/Selective-Loading of entity properties and many more.

Written by Animesh

June 30, 2010 at 7:12 pm

Posted in Technology

Tagged with , , , ,