Kafka vs RabbitMQ vs Enduro/X vs ActiveMQ on Go – performance benchmark of the year 2018!

This blog explores to compare with equal tests large quantity of the modern middlewares available on the market. The test compares most trending middlewares which are:

The testing is based on Enduro/X middleware benchmark suite. Which basically performs two types of tests between to processes located on local machine. The tests are following:

  1. Two way RPC (client call server and server responds back)
  2. One way – client process connects to server and streams all the data to Server

The testing code is written for Go language and is based on Enduro/X Bechmark Suite.

Test platform

Both benchmarks ar performed on following system:

  • Linux Mint 18.3
  • Linux 4.10.0-38-generic #42~16.04.1-Ubuntu SMP Tue Oct 10 16:32:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • CPU: Intel(R) Core(TM) i5-4300U CPU @ 1.90GHz
  • RAM: 12GB

Java version 1.8:

$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)

Test results

This section lists test results got from the system testing.

RPC Style

vs

RPC style calls – Enduro/X vs RabbitMQ vs Kafka vs ActiveMQ

From the results we see that Enduro/X performs very well with RPC style calls. All other middlewares keeps almost the same results. Seems like ActiveMQ gets little better with bigger message sizes.

One way (classical publish/subscribe)

vs1w.png

One way calls: RPC style calls – Enduro/X vs RabbitMQ vs Kafka vs ActiveMQ

With one way calls. We get in the front the Apache Kafka. Then Enduro/X follows. all other competitors seems to run at the same level. Also we see that with bigger message sizes, at some points ApacheMQ gets in the front.

CPU/Memory Usage during testing

This section lists CPU usage author monitored on his testing machine.

Kafka

kfk_ok

CPU Usage Kafka RPC

kfk1w

CPU Usage Kafka Oneway

RabbitMQ

rbt

CPU Usage RabbitMQ RPC

1wrbt

CPU Usage RabbitMQ Oneway

Enduro/X

ex

CPU usage Enduro/X RPC

ex1w

CPU usage Enduro/X Oneway

ActiveMQ

amq

CPU Usage ActiveMQ RPC

1wamq

CPU Usage ActiveMQ Oneway

System stability

Author have faced following issues during the testing:

Kafka notes

For Kafka, Author faced “Queue full” error during the one way publish to topic test. The quick-and-dirty fix was to restart client call during the facing this error.

ActiveMQ notes

During the ActiveMQ testing with Go (uses STOMP protocol), author got several times, “connection closed” during the client send operations, for example:

$ ./run1w.sh 
~/projects/endurox-benchmark-suite/tests/activemq ~/projects/endurox-benchmark-suite/tests/activemq
Starting server process...
panic: send on closed channel
goroutine 1 [running]:
github.com/go-stomp/stomp.(*Conn).Send(0xc42009e3f0, 0x563f5d, 0xd, 0x566378, 0x18, 0xc4200b4a80, 0x940, 0x940, 0xc42004dbd0, 0x1, ...) /home/mvitolin/projects/endurox-benchmark-suite/src/github.com/go-stomp/stomp/conn.go:437 +0x299 main.runbench.func1(0xc42000e038, 0x14c0000000388, 0xc4200b4a80, 0x940, 0x940, 0x1, 0x0, 0x0, 0x0, 0x0)
 /home/<hidden>/projects/endurox-benchmark-suite/src/amqclt/amqclt.go:51 +0x17f exbench.Ndrx_bench_clmain(0xc42000e038, 0x1, 0xc42004dec8, 0x2d)
 /home/<hidden>/projects/endurox-benchmark-suite/src/exbench/testutil.go:248 +0x345 main.runbench(0x0)
 /home/<hidden>/projects/endurox-benchmark-suite/src/amqclt/amqclt.go:38 +0x19f main.main()
 /home/<hidden>/projects/endurox-benchmark-suite/src/amqclt/amqclt.go:108 +0xd7 amqclt -num 400000 -oneway failed
Test exiting with: 2

 

Thus the code was fixed to re-connect in case of client errors.

 

Conclusions

During the testing the results shows that Enduro/X shows goods results with RPC style calls. For Oneway testing (publish only), Apache Kafka was the best. But downside for Kafka is that calls were not blocked, thus “dirty” solution was to restart the sending message to the queue while the Enduro/X no “dirty fixes” are needed, as when the queue is full, the process blocks and waits for free space. Then after all the ActiveMQ with Go STOMP client was also showing good results when message size increased. But the downside with STOMP client was that, during the testing, the client got disconnected for some reason. RabbitMQ also seems to be stable platform and no special fixes were needed to execute the test cases (except that channel send must be used to get “blocked” style calls). RabbitMQ showed quite good stable results.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s