Richard Bucker

GoLang Message Queues

Posted at — Jul 31, 2014

Approximately 4 years ago I designed and build a payment gateway in Python. The APIs were exposed to the POS devices as REST calls and the outbound request was either a standard TLS socket or HTTPS to the acquiring processor. The REST components were implemented using a standard event driven async webserver and the worker side was implemented as a collection of multiple worker processes each processing one connection and one transaction at a time. In the middle of the two was a transaction broker that handled the queueing as well as impedance correction between the two state machines.The transaction was parsed, converted into a ctx (context) and stored in a redis hash as quickly as possible. A UUID was assigned to the transaction and was the only part of the transaction (the key) that was passed from function to function; and if that function needed some data then it pulled it from redis directly. Finally the transaction ctx was passed around using ZeroMQ.Over the last few weeks I have been evaluating go chan and things have been going nicely. Channels are a nice model especially when considering the flow based programming model. The challenge, however, is that the channels are not network enabled and if you want them to work over a network then you either need a library or you need to implement them yourself.The bigger challenge is that golang networking is where the data structure to be passed needs to be marshaled into some payload that is easily transmitted and on the other end reconstituted. And so now you get into the protbuf, msgpack, json, bson, xml argument. In a client/server pair the message needs to be processed 4 times. In a client/server/gateway system the payload is processed 6 times, and in a client/broker/server/gateway system the payload is processed 8 times.When calculating Big-O values for some algorithm most computer scientists compute the number of comparisons based on some trivial “n”. As in integers or strings or some arbitrary structure; and while “n” is the number of compares of the object when you determine the actual number of comparisons based on the number of simple objects (ints) things devolve quickly. When “n” is an object then the number of comparisons are the same regardless of the type. But when determining the actual amount of work that the string version will perform it is some multiple of “n” based on the average length of the string or average size of the struct.Lessons learned (a) process the object as little as possible. (b) move the object as little as possible. (c) transport a proxy for the object instead.As for GoLang there are a number of MQ options:ZeroMQ/crossroads - implemented in C++ and although there are several integrations it’s not as solid as the python versionlibchan - not sure what the state of the project. It seems to be a lynchpin for docker but the code has not been updated for a while and the documentation and test cases puts a burden on the dev. The travis instance is not running. Poorly documented. “like channels” is is wrong.mbxchan - a waste of time. The project is not an idiomatic workspace. the installation instructions never refer to GOPATH or GOROOT. They never specify the pwd so that you know where things are relative to GOPATH etc.nats - just released. Can’t say much yet. The documentation is weak and there is a case where they carried a ruby install script forward. One good point is that the docker version is close to the current rev except they do not include the dockerfile in the source. While they have a github account and both the gnatsd and nats projects are present their homepage nats.io has a separate download page. This is not very idiomatic since go get and go install are the preferred methods.nsq - the docker images are old. The Dockerfile is non-existent. Shame on me but I should probably know more about this because a good docker instance is needed and of great benefit.nanomsg - one of the things I liked about nanomsg is that it supports multiple topologies. Fan-Out, fan-In, etc…netchan - this is the old deprecate netchan library from the golang authors. It had not been replaced yet. What’s nice about this model is that it really works.  The actual payload is abstracted away from the user and the user code is strictly limited to the channels and configuring the export/import. The downside of this model is that only one worker can exist on the same port on the same machine. Fan-Out routing requires multiple ports or hosts.There are several choices… none of which would compete with the python/ZMQ version but I think I should have removed ZMQ and gone bare metal redis.UPDATE: que-go seems to be contender. My only hesitation is that it comes from a ruby design and while that is a positive feature if I were migrating apps… it’s not necessarily a good idea for a greenfield in which case I’d prefer to try gnatsd from Apcera.