JavaEE 6: asynchronous contexts and Ajax push

Posted on 29. Sep, 2010 by Guilherme Silveira in java

Backend developers requireing high scalability already know the no blocking IO tricks for a while: selector, poll and epoll in Linux, kqueue in MacOS and BSD, finally I/O Completion Ports in Windows. A Renato Lucindo session at QConSP touched questions and solutions that can be implemented using those ideas. Those standards and implementations are already old but are coming back stronger than ever. But how come non blocking IO and connection strategies can help a typical web application in its every day life? Reverse Ajax is the first answer.

Consider the trivial example of a bidding system where we are required to update a products price every time a new valid bid comes in. A simple strategy would be to implement client browsers sending an Ajax request every 5 seconds and, when there is any updates to the price, the server answers with its new value through a javascript response. This is the most basic reverse ajax implementation, polling, providing an user impression that the value is always up to date. The drawback is that stale data might be shown, which is acceptable for most applications.

For something more sofisticate, we need to push data from the server: as soon as there are data, we send it to our client. Many refer to Ajax push as comet, which are the most used techniques: long polling while the server holds the request open for some seconds until it there is the first pack of data to send; or creating a streaming, keeping the connection open for as long as necessary, sending new data as it comes. In both cases we can easily implement the client, but what about the server?

In the server, we need to keep an open socket with the client. In a naive implementation, there would be for each HttpServletResponse, a related thread in waiting mode. Whenever a new event is fired, a notify/notifyAll is necessary (otherwise one can make use of BlockingQueues). This implementation, known as thread-per-request is expensive on Java, because of its memory footprint and the use of native threads by the JVM. When the number of clients grows, the number of threads will become a bottleneck on the systems overral performance and scalability, even if all connections are idle, just waiting – a quite common case in our scenario.

Im many cases the best approach would be to use a few (or just one) threads that execute the event loop and check for a SocketChannel from a Selector for its read/write ready state, and execute the related operation whenever necessary. Implementing that with java.nio is not difficult and, above all, the servlet containers implemented this strategy and provide abstractions simpler to handle event loops in state machines. Tomcat 6 already came with an abstraction for Comet with several callbacks for each socket state, while Jetty 6 already did the same through a limited implementation of continuations. Both had the idea to provide a way of suspending a request and while the processing was suspended, the thread would not go into a blocked state, being able to handle other requests that had sockets in a read/write ready state.

Since november 2009 we do not need to use a servlet container specific solution: the Servlet 3 specification supports what was called asynchronous contexts. If one needs to push data to all connected clients, it suffices to notifies clients that the request will not end by the return point of the service method (and, therefore the doGet and so on). We do this by invoking req.startAsync(). After that, the returned context can be added to a collection in order to broadcast messages to all waiting users.

You can run the above test with your servlet container (Tomcat 7 e Jetty 8 already support it). Just configure your servlet through annotations so it handles asynchronous calls:

By doing a request to /subscribe you will notice that the browser will keep waiting for a response, as the server did not finish it. As soon as our application is ready, we will send a push to all waiting clients (the ones that requested “/subscribe“). Lets change the /subscribe post request to send a message to all registered clients, adding this message to the end of our queue, with its own id:

There is a need for a responsible object that listens to new messages and, if positive, send the enqueued messages to the clients contained in our AsyncContext list. We will do it on a Thread that is fired during our servlet initialization and waits for new messages on a BlockingQueue.take blocking method invocation:

I have removed all checked exceptions handling issues for legibility, while the entire ChatServlet can be found here.

But how can we do a test with thousands of clients? This can be an issue if you use any library working with a thread-per-request strategy, not being able to start many simultaneous clients, giving the excessive thread count. Your client will also need to use pure java.nio, or even an http library that uses this API. This is the case with the newest do Apache Http Core, and here is a simple example.

You will be impressed by realizing that the machine at your home is capable of supporting more that 5000 clients with Jetty 8, only hitting this number due to the limit of files open in your operating system.

There is always the need to save expensive resources such as threads, connections and file descriptors. Using Servlet 3 one can balance those resources, minimizing memory consumption and processing, paying extra on open descriptionrs, which helps a lot when doing reverse ajax with push.

Many thanks to Renato Lucindo and Gleicon Moraes for the lessons, talks and corrections.

Tags: ajax, ajax push, async, asynccontext, comet, io, io não-blocante, nio, nio2, selectors, servlets