Richard Bucker

The Network Is the Machine : Erlang is not all that

Posted at — Jun 21, 2011

I like erlang and I like it most because it solves a number of problems, however, the problems that I think it solves in general application development are not the kinds of problems that most erlang programmers want to solve. For the [sp: life] live of me I cannot understand why erlang programmers would implement a database like Riak. It’s a complicated undertaking and frankly considering how deep the callstack has to be at times it does not seem practical without a real debugger.As I consider the amount of work that it takes to implement a single credit card transaction I realize that the entire callstack is going to consist of a few thousand instructions regardless of the language. The hardest part of a credit card transaction is the DB record versioning and not the actual in-memory workflow.So then we start talking about the threading and IPC. MEH. I no longer care about that stuff. Not even for a second. With libraries like ZMQ “we” should reconsider how we allow processes to communicate. Modern MQs are fast, reliable, persistent, and easy.Finally, If you have a transaction that takes a predictable number of machine cycles (specially in the context of a CC transaction) then executing the transactions synchronously via a fixed number of workers will have less overhead than each transaction being launched at the same time. As light as they may be there is still overhead. O(1)+1 still has a +1 and at some time…. say after 1M transactions they will count for serious performance.[update 2011-06-22: When the machine is idle then parallel execution and all the thread happiness in erlang makes sense, however, when the machine is busy then single process execution makes the most sense as there is no overhead no matter how small. ie; if you have 100K transactions you still have 100K worth of work to complete. The mean execution time will be higher just because of the latency and overhead but on the hole there is no real advantage. see nodejs as a partial example.][update 2011-08-24: I get it and it was not because I was talking to the CTO at Riak. Two days ago I was looking at the connection pool to a Postgres DB. That’s when I realized that an erlang implemented database… at least for socket(s) and systems with long runtimes like connection pooling; was a very strong use-case for erlang.]