Richard Bucker

Updated Thoughts on Flow Base Programming

Posted at — Feb 26, 2015

My implementation of a flow based framework has been stalled for a few months. ┬áIt’s nothing serious but a refactoring is planned. I’ve also run into a few new learnings that have triggered more ideas.(I’ve already mentioned FlowHub and NoFlo so I’ll leave them out).GoFlow has a good design and uses some of the same ideas that I published some time ago. In this case I’m referring to anonymous structure attributes. One thing I think might make it more extensible would be moving the channels to the structure instead of hiding that info. There is a strong separation between the node and the node graph. In my implementation I used a JSON file to define the network structure where GoFlow uses straight configuration as code.Following up I’m also interesting in gnatsd. Gnatsd is a message queue broker and client library. What makes it outstanding is it’s implementation of netchan. The netchan was initially a Go core library in the stdlib but was later moved to the experimental repo and later deprecated. There have been other; similar; implementations but there are two that are notable. (a) from the docker team, (b) from gnatsd.I have been thinking of experimenting with Go’s native RPC structure. Since it runs on the http rails I’m hoping that it’s going to be concurrent. But one project that might step up the game a little is gRPC. This project extends RPC with pluggable protocols and a performance gap with similar libraries. While gRPC supports multiple client languages my Flow implementation is limited to a a global namespace that might have minor remote services. It is possible that the services could be implemented in other languages but unlikely.