Richard Bucker

consensus says wtf?

Posted at — May 4, 2015

Just how many consensus based replicated key/value storage systems do we need? Clearly this has become the latest hello world or todo application that everyone implements on their way to whatever is next… but is it really necessary considering that all of these implementations are already open source and most, if not all projects, will accept pull requests.Here’s my list:etcdconsullogcabinchubbyzookeeperblockstore (???)And that’s just what I can remember.  Follow this link for a list of about 50 implementations. I’ll admit that there is something to be said for choosing different programming languages and some features but still… at some reasonable point we need to elect just a few masters so that we can all go back to the task at hand. (Embedding this tool deeper into the stack)I suppose that since raft and paxos are relatively new the next generation of PHd’s need something to tinker with. But seriously, at some point we need to take it to the next level.PS:  I realize that logcabin is from one of raft’s inner circle but there is no accounting for taste in the tools selection. For example it depends on scons which has not been updated in 8 months. I cannot imagine why the git version is a dependency. Looking at the code there are a number of things that could be challenged but that would be petty when I think the bigger issues have nothing to do with this project.I suppose there is an interview question in here somewhere:design or implement a raft protocolwhere does it failwhat’s it good forcan it be used for general purpose replication or key/value only