Richard Bucker

JSON and more JSON

Posted at — Mar 10, 2012

A few articles ago I wrote about processing CDRs from an asterisk server. One of the design decisions I made was using very simple JSON messages as the message payload from the publisher to the subscriber. As most are familiar with JSON it’s a key/value container represented in UTF-2/ASCII. The format of the message is well documented and widely implemented.There are a number of alternatives to JSON, like BSON, s-expr, msgpack and a few others. I settled on JSON because it was trivial to hand code a JSON payload without having to make any decisions and that it mapped to/from hash datatypes in python and ruby.In the article I wrote the publisher in C and the subscriber in Ruby.The publisher was an not an easy decision but by the time I committed to C I had a plan and it’s just one page of code for JSON, Redis and a but of parsing. (this code had to be fast and small for the reasons discussed in the article)The subscriber was a different. It did not have to be all that fast… but in hindsight I’m thinking that the Ruby code worked as a proof of concept and now I want to reduce the memory footprint of the subscriber, however, as I look at the output of “top” I see that it has consumed only 54K of memory.  Not bad. I’ll have to watch it over time to see of there are any leaks.All that aside, I was thinking that I would implement the subscriber in C. I knew that one of the complications of  JSON+C was that C did not have a native datatype that the JSON could be parsed into. So the code would have to work more like strtok().  I supposed that’s not much of a problem but it’s going to add code on my side to implement. Big deal- 20 or 30 lines of code.But what was also interesting was the wide variety of implementation in the different libraries that the JSON project links to. Some of the projects were small and some were outrageously huge. I suppose there is more than one way to skin a cat. At least in this case while the intellection pursuit is interesting that’s about it. The Ruby solution was memory friendly for the moment.