Akka vs. Node

2015-01-20

This post is about the two competitors Akka (akka-http respectively) and NodeJS and their abilitiy to scale. To create kind of a real word comparable benchmark, I have created the following scenario (the sources can be found on GitHub:

  • There is a MongoDB database containing 10,000 user records (each consisting out of an id, a name and a description string)
  • Both, akka-http and nodejs provide a single route /api/user where they create a query to MongoDB, that extracts randomly one user record and return it as JSON string
  • This route is tortured with wrk with different levels of concurrency

Having some kind of external IO service within the benchmark is intentional. Pure CPU/memory bounded REST services are not the usual real world thing and quite theoretically. Note, that for both server there is a time window in each request, where they have to wait for MongoDB to respond - hence they can show their ability to use this spare time for handling other incoming requests.

The benchmark execution is done by the following recipe:

  1. Run both the Akka and the NodeJS server
  2. Warm both servers up by accessing them both for several seconds
  3. Start with a client request concurrency of 1
  4. Access the route on the Akka server as fast as possible for one minute
  5. Sleep a little
  6. Access the route on the NodeJS server as fast as possible for one minute
  7. Sleep a little
  8. Double the concurrency level and repeat with step 4 until a concurrency level of 256 has been reached

I have run this benchmark on my MacBook Pro Retina 15” Mid 2014 (having a quad core Intel-i7 and 16 GiB of RAM). Obviously it would also be very interesting to have this benchmark running on a server machine with more cores available.

Results

Raw results can be found here.

Requests per second

This chart shows for both competitors who many requests they can handle per second depending on who much concurrent requests there are at a time.

For low concurrency NodeJS beats Akka by a fair amount. But spinning up the pressure Akka scales much better until it reaches it’s saturation level. At this stage Akka handles about 54% more request per time as NodeJS does.

Response times

The response times are also a good metric to compare. The next graph shows the average response time:

Having no concurrency both are really fast with response times below 5 milliseconds. Bringing concurrency in the average response time of Akka “sky rockets” up to around 45 milliseconds while NodeJS just raises a little bit. This shows Akka’s needed overhead to manage real concurrent threads while NodeJS is just an event loop, hence single threaded. At a concurrency of 8 Akka’s response time drops again by 50% (could have to do with the fact, that my MacBook has 4 cores with hyper threading). As concurrency goes up the response time of NodeJS raises proportionally. Akka again scales better until the point, where it beats NodeJS.

The last graph shows the 50%-quantile of the response times. That means, that is shows the response time in which 50% of all requests were handled:

We see: Even if NodeJS is better in average response time (for low/medium concurrency), Akka still manages to serve 50% of the requests faster than NodeJS most of the time regardless of the level of concurrency.

Conclusion

This is short: Planning to build a big REST application? Use Akka! It just scales way better and allows to use every single bit of computation power you have.


Comments: