Richard Bucker

Message Queues and nothing but Message Queues

Posted at — Sep 21, 2011

[Update 2011.09.21] The ink had hardly dried on this post when I decided to quickly evaluate gearman. It’s immature and direction internally seems dizzy. So pass on this one too.I’m reading up on MQs again while I’m waiting for a conference call. I do not want to disrupt my code before the demo (rule #1 of demo-club). So I’m making yet another list of all of the MQs out there. ActiveMQ RabbitMQ Amazon SQS Google Queue/Task Gearman ZeroMQ Sparrow Starling Kestrel RestMQ Oracle Advanced MQ IBM Websphere MQ MicrosoftMQ JBoss Messaging Sun Open Message Queue Apache QpidThere are several ways to compare these MQs. Core source language, client language support, implementation detail, inspiration or initial design, current activity, licensing, cost, deploy OS, performance/TPS, use-cases, dependencies, deep dependencies, mindshare.So here is that list again. ActiveMQ - apache and meant for Java JMS although there are libs for other languages it really depends on the actual message payload. RabbitMQ - build with erlang, while it works well and is the MQ for ejabberd it’s pretty heavy weight. Amazon SQS - it’s not free or at least cost effective. They have had huge outages. For security reasons this has to be onsite. Google Queue/Task - For security reasons this has to be onsite and only supports java, python and possibly go. Gearman - There is potential here. It seems to have been rewritten in C from perl. It is open source and free to use. They appear to be active and there is a CLI making access easy. This is worth looking into. (the only downside is that it’s written in Java) ZeroMQ - 0mq leaves a lot to the developer or architect to decide and fill in the blanks. My original implementation used beanstalkd and it was razor fast and trivial to implement. 0mq has plenty of gotchas and the burden is on the developer. I’d use it again but it’s not going to be #1. Sparrow - ruby? This is just not going to make the list. Starling - another ruby implementation. Kestrel - argh… this one is implemented in scala. So you get all of the non-functional java components (unverified) and long namespaces. RestMQ - I like the pythonic implementation, however, while twisted is a solid project it appears to be conflicting with my main project based on tornadoweb. Oracle Advanced MQ - not free IBM Websphere MQ - not free MicrosoftMQ - not free JBoss Messaging - too many dependencies. This is akin to J2EE’s JMS. And for that I might as well use ActiveMQ. Sun Open Message Queue - They do not live here any more. Apache Qpid - supports AMPQ and trying to get 100% compliant. I suppose this is interesting in that disparate systems will be able to communicate. Looking at the source tree there is just so much code and since I do not want to dedicate this kind of time to it… I think it’s an easy PASS. beanstalkd - The last release was over a year ago. I’ve posted on their group hoping to get some sort of answer. There has been some activity on their website. The APIs have a TTR novelty API that I like. Specially in my fire and delete application.One of the novel things about the apache team is that when they acquire a technology it’s long before there is a wrapper around it. For example Apache Camel.So here is the state of things. RestMQ - I’m going to continue my attempt to rewrite RestMQ in perl using Mojolicious. If I get some decent results then I’ll publish them/it. Gearman - deserves some immediate investigation. I’m hoping to make it through the painful documentation. Beanstalkd - while it has not had much recent action I’m hoping to see some info in my inbox shortly. I like their broker and while they do not specifically support req/resp it’s not really needed and can worked around. ZeroMQ is just an API. My broker is getting more and more complicated. Message routing is also getting complicated.