This is just a note to my future self. I just finished watching the last 10 minutes of a goto; presentation that I started a few months ago and it reset a few of my core architecture beliefs and has be going on the bath of thinking outside the box again. While I agree that distributed systems with heavily co-mingled semantics need protection whether it’s locks or some other mechanism. But if you can partition the work into it’s simplest form it’s likely that (a) locks are not needed (b) the simplest implementations will reduce failure and when there is a failure it’s easy to fix.
I’m thinking about an application design that uses the filesystem and filesystem replication instead of DB replication for keeping systems in sync; in contrast to using a relational system. Many of the requisites for an RDBMS are also available in the basic filesystem. There is a reality that if the RDBMS is not used the right way then the ACID features of the “system” are degraded, however, implementing a full ACID system will also reduce the efficiency/throughput of the “system”.
This might sound like nonsense except that these are notes for myself and it’s based on experience. For example in a payment gateway the host-capture subsystem from the POS perspective is pretty simple. Store the request and the response with some basic telemetry. Choose the one path that makes the most logical sense for the transaction and replicate the data across all nodes.
When a transaction arrives that requires information from a previous transaction there is enough information in the second transaction to locate the first as a hash function O(1) in the filesystem. It’s no more or less complicated than a relational system except that it does not require a separate DB that is itself replicated.