animesh kumar

Running water never grows stale. Keep flowing!

Posts Tagged ‘Guice

Jesque-Guice binding

with 2 comments

In our Java stack, we use Guice IOC library quite heavily. We are kind of smitten with it. Now, when we needed a task queue to process our background jobs, we settled with Jesque which is a port of Github’s Resque project, and runs on Redis which was already part of our stack. In the process, we also added delayed task functionality to this.

Problem

Anyways, the problem we were facing was with Injecting Guice dependencies into Jesque workers. Our workers performed heavy operations and at times had to read-write from DB, or speak with other services.

Jesque uses Java reflection APIs to instantiate and run its workers (which are Runnable classes) and so, it becomes very difficult to inject Guice managed objects and services into Jesque workers.

We started the hacky-and-ugly way by creating static references to relevant services that our workers needed. if it were for few services, this would have worked, but our workers kept growing in features and reach, and after a while the whole code base was stinking. We had to do something about it. Something elegant!

Solution

So, we created Jesque Guice binding project. You can annotate your worker classes, and Guice will then discover, register and start them. Let me show you some code.

First, let’s create an ExampleWorker. You have to remember that the Worker must implement Runnable interface, and it must be annotated with @Worker which Guice uses for discovery and binding.

// ExampleWorker

@Worker(job = "ExampleJob",                 // job name
        queues = { "EXP_QUEUE" },           // queue names
        enabled = true,                     // enabled
        count = 1,                          // 1 instance of this worker running
        events = { WorkerEvent.JOB_SUCCESS, WorkerEvent.WORKER_START }, // Events to listen to
        listener = EchoListener.class      // WorkerEventListener
)
public class ExampleWorker implements Runnable {
    // LOG
    private static final Logger LOG = LoggerFactory.getLogger(ExampleWorker.class);

    // Note: Only field level injection would work!
    @Inject
    ExampleService service;

    String arg1;
    String arg2;

    // Must keep an empty constructor for Guice to discover this
    public ExampleWorker() {
    }

    public ExampleWorker(String arg1, String arg2) {
        this.arg1 = arg1;
        this.arg2 = arg2;
    }

    @Override
    public void run() {
        LOG.info("Running worker={}, with arg1={}, arg2={}", new Object[] { getClass(), arg1, arg2 });
        // calling service
        service.serve();
    }

}

@Worker attributes let you control the behavior. The above worker, ExampleWorker, listens to EXP_QUEUE queue, accepts Jobs by name ExampleJob, has an WorkerEventListener defined by Guice Managed class EchoListener which listens to WorkerEvent JOB_SUCCESS and WORKER_START. There is only ONE instance of this worker running.

Note: ExampleWorker has been field @Injected with ExampleService. Please remember, that
Only field @Inject will work with Workers, because Jesque uses constructors to pass Job arguments.
– Also, you must keep an empty constructor around so as to let Guice discover this worker.

Now, since we have added a Listener, we must define it.

// EchoListener

public class EchoListener implements WorkerListener {
    // LOG
    private static final Logger LOG = LoggerFactory.getLogger(EchoListener.class);

    @Override
    public void onEvent(WorkerEvent event, Worker worker, String queue, Job job, Object runner,
            Object result, Exception ex) {
        LOG.info("onEvent ==>> queue={}, event={}", new Object[] { queue, event });
    }

}

For the sake of demonstration, this has been kept very minimal. But, mind you, you can @Inject any Guice managed objects into this, using constructor and/or field injection.

Now, let’s define the ExampleService that we want to @Inject into our worker.

// ExampleService

public class ExampleService {
    /** The Constant LOG. */
    private static final Logger LOG = LoggerFactory.getLogger(ExampleService.class);

    public void serve() {
        LOG.info("Heya! I am not here to serve, just to show you how injection works.");
    }
}

Wonderful! Let’s now bind these all together in a GuiceModule.

// ExampleModule

public class ExampleModule extends AbstractModule {

    @Override
    protected void configure() {
        // Jesque Guice
        install(new JesqueModule());

        // Jesque Client
        Config config = new ConfigBuilder().withHost("localhost").withPort(6379).withDatabase(0).build();
        bind(Config.class).toInstance(config);
        bind(Client.class).toInstance(new ClientImpl(config));

        // Worker
        bind(ExampleWorker.class).asEagerSingleton(); // Must be singleton
        // WorkerEventListener
        bind(EchoListener.class).in(Scopes.SINGLETON);
        // Worker Executor (This is where they actually run)
        bind(WorkerExecutor.class).to(SimpleThreadBasedWorkerExecutor.class);
        // Service (will be injected into workers)
        bind(ExampleService.class).asEagerSingleton();
    }

}

Here, first we install JesqueModule, and then bind other objects.
– Bind Jesque config and client.
– Bind Worker, Lister, Service etc.

You will notice we have also bound WorkerExecutor. This interface accepts net.greghaines.jesque.worker.Worker instance and runs that on a thread. Jesque-Guice comes with 2 simple implementations:

1. SimpleThreadBasedWorkerExecutor which run each net.greghaines.jesque.worker.Worker on an unmanaged separate thread, and
2. CachedThreadPoolBasedWorkerExecutor which creates a CachedThreadPool where net.greghaines.jesque.worker.Worker is run.

You can implement your own strategy or provide your own ExecutorService.

Run

Now we have everything, let’s run it then.

// Main

public class Main {
    /** The Constant LOG. */
    private static final Logger LOG = LoggerFactory.getLogger(Main.class);

    /**
     * @param args
     */
    public static void main(String[] args) {
        Injector injector = Guice.createInjector(Stage.DEVELOPMENT, new Module[] { new ExampleModule() });

        // Get Jesque client
        Client client = (Client) injector.getInstance(Client.class);
        LOG.info("Publish jobs");
        // Push jobs
        client.enqueue("EXP_QUEUE", new Job("ExampleJob", "hello", "job1"));
        client.enqueue("EXP_QUEUE", new Job("ExampleJob", "hello", "job2"));
    }
}

You should see,

// Job - 1
DEBUG c.s.commons.jesque.GuiceAwareWorker - Injecting dependencies into worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6b754699
DEBUG c.s.commons.jesque.GuiceAwareWorker - Delegating to run worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6b754699
INFO  c.s.c.jesque.example.ExampleWorker - Running worker=class com.strumsoft.commons.jesque.example.ExampleWorker, with arg1=hello, arg2=job1
INFO  c.s.c.jesque.example.ExampleService - Heya! I am not here to serve, just to show you how injection works.
INFO  c.s.c.jesque.example.EchoListener - onEvent ==>> queue=EXP_QUEUE, event=JOB_SUCCESS

// Job -2
DEBUG c.s.commons.jesque.GuiceAwareWorker - Injecting dependencies into worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6602e323
DEBUG c.s.commons.jesque.GuiceAwareWorker - Delegating to run worker instance = com.strumsoft.commons.jesque.example.ExampleWorker@6602e323
INFO  c.s.c.jesque.example.ExampleWorker - Running worker=class com.strumsoft.commons.jesque.example.ExampleWorker, with arg1=hello, arg2=job2
INFO  c.s.c.jesque.example.ExampleService - Heya! I am not here to serve, just to show you how injection works.
INFO  c.s.c.jesque.example.EchoListener - onEvent ==>> queue=EXP_QUEUE, event=JOB_SUCCESS

Works… eh? 🙂

You can grab the project source at github https://github.com/anismiles/jesque-guice Give it a try. I hope this will help some of you folks. Share your thoughts with me.

Happy hacking!

Advertisements

Written by Animesh

September 18, 2013 at 4:44 pm

Jetty Executable Webserver Archetype

with 2 comments

We create and deploy many rest based API apps on our servers. Now, instead of managing multitudes of webservers on different machines, we make our deployable archives wrapped with all the necessary dependencies in order to be self-executable. Benefits?

  • We just have to ensure that all our servers have the same Java environment, and no other dependencies.
  • No waste of time configuring, managing web (tomcat, jetty etc.) servers.
  • And, we only have to distribute a single executable.

Hassle free management. Won’t you agree?

To cut down on bootstrapping time, we have created a maven archetype to help us quickly create and start on new API projects. Our tech stack is Google Guice, JBoss Resteasy, Logback, and Embedded Jetty.

Github link: https://github.com/anismiles/jetty-webserver-archetype

Install

You can either build/install from source

git clone https://github.com/anismiles/jetty-webserver-archetype.git
mvn clean install

Or you can simply download the jar and install that.

mvn install:install-file \
-Dfile=jetty-webserver-archetype-1.0.jar \
-DgroupId=com.strumsoft \
-DartifactId=jetty-webserver-archetype \
-Dversion=1.0 \
-Dpackaging=jar \
-DgeneratePom=true

Usage

Let’s say you want to create a new project “hello-world” with group “com.hello.world”, you would run:

mvn archetype:generate \
-DgroupId=com.hello.world \
-DartifactId=hello-world \
-Dversion=1.0-SNAPSHOT \
-DarchetypeGroupId=com.strumsoft \
-DarchetypeVersion=1.0 \
-DarchetypeArtifactId=jetty-webserver-archetype

That’s it. You are ready to roll. You have set up a basic working API app. Now, you can run it in dev mode

mvn clean jetty:run

Or in production mode

mvn clean package
java –jar target/hello-world-1.0-SNAPSHOT-dist.war start &

To check, open http://localhost:8085/
To stop:

java –jar target/hello-world-1.0-SNAPSHOT-dist.war stop

Further, you can pass additional Jetty, Logback or your App’s properties

java \
-Djetty.configurationFile=<jetty-config> \
-Dapp.configurationFile=<app.properties> \
-Dlogback.configurationFile=<logback.xml> \
–jar target/hello-world-1.0-SNAPSHOT-dist.war start &

They will override default properties setup by the executable.
Ideally you might want to create an init.d file to start and stop you API app efficiently. Here is a prototype:

#!/bin/sh

# Executable war file
WAR_FILE=/var/apps/<my-war-file>.war

# Configuration files
APP_NAME=<app-name>

JETTY_CONFIG=/etc/${APP_NAME}-jetty.properties
APP_CONFIG=/etc/${APP_NAME}-app.properties
LOGBACK_CONFIG=/etc/${APP_NAME}-logback.properties

# Process
PIDFILE=/var/run/${APP_NAME}.pid

# Java arguments
JAVA_ARGS="-Djetty.configurationFile=${JETTY_CONFIG} -Dapp.configurationFile=${APP_CONFIG} -Dlogback.configurationFile=${LOGBACK_CONFIG}"

# Command
CMD=$1

if [[ -z ${CMD} ]]; then
  java ${JAVA_ARGS} -jar ${WAR_FILE} usage
  exit 1
fi

if [[ ${CMD} == 'start' ]]; then
    if [[ -f ${PIDFILE} ]]; then
        echo "Already running"
        exit 1
    fi

    if [[ -f ${WAR_FILE} ]]; then
        echo "Starting jetty: ${WAR_FILE}"
        java ${JAVA_ARGS} -jar ${WAR_FILE} start &
        PID=$!
        echo "$PID" > ${PIDFILE}
        echo "Started ${APP_NAME} with pid: ${PID}"
    fi

elif [[ ${CMD} == 'stop' ]]; then
    # Try gracefully first
    java ${JAVA_ARGS} -jar ${WAR_FILE} stop
    sleep 10
    if [[ -f ${PIDFILE} ]]; then
        PID=`cat ${PIDFILE}`
        test -z $PID || kill $PID
        rm ${PIDFILE}
        sleep 1
        echo "Forcibly Stopped ${WAR_FILE} with pid: ${PID}"
    fi

else # Just let the other cmds through...
    java ${JAVA_ARGS} -jar ${WAR_FILE} ${CMD}
fi

exit 0

That’s it! have fun. 🙂

Written by Animesh

January 8, 2013 at 1:48 pm