Clustering/High Availability/stream's multi publishers

Question

Hi,

While digging in robustness and taking advantage of node's potential automatic content mirroring feature, I would like to submit a test-case here. Assuming we have 3 nodes, one being the 'master' one, all others being 'slaves' ones (through -initprivkey and -autosubscribe option on multichaind command line). So, all slaves nodes subscribes automatically to stream created on master and reverse (when a value/stream is added on a slave node). Square situation. If one node is down, even badly (hence no more wallet file etc...), a remaining 2 slaves configuration, or one master-one slave configuration can continue working (assuming one have some top API/mechanism balancing seamlessly in-between available nodes (e.g. somehow one of the remaining nodes becomes the newly 'node of contact/entrance' etc...)).

Now, let's consider the addition of publishers notion to streams in such multi 'clone' nodes configuration, in order to manage some read/write access to streams (one stream == one publisher (e.g. address/wallet)). So let's say one address is created on one node (local, not shared with other nodes, normal situation), a new stream created on the same node (distributed to all nodes, the new streams is visible in all nodes), data published to that stream on behalf of the newly created address (publishfrom - distributed to all nodes, the new data is visible on all nodes, including the new address 'broadcasted/recorded' as the data's publisher). But, if the node where the new publisher was created crash badly, other nodes can not continue publishing on behalf the new publisher because of the lack of the new publisher's private key (which was lost in the node's crash).

Hence my question: what are my options here if I want to be able to publish to the new stream on behalf of the new publisher from the other (remaining) healthy nodes (like if no crash occurred)?

I would like/expect here some MultiChain 'built'in' options, not external one based on some backup feature (dumpwallet...), in order to transmit private key... which is not something good to do... I guess. My silly mind may imagine storing private keys in a dedicated stream (which will be automatically and naturally transmitted to all nodes - e.g. one address is created on one node, the associated private key is pushed to some dedicated stream). Then, when a node try to publish on behalf an address it does not know locally (because the address was created on another node), it searches in the dedicated stream for the associated private key, use the importprivkey function and then publish the data to the stream... seamless from user point of view, no file/backup manipulation etc...

What do you think? How silly/dangereous is that (using a stream to automatically store/broacast/transmit/propagate private keys among nodes in order to ensure high availability)? Any other option? Fire at will.

Thanks.

Answer 1

This sounds like an x-y problem.

You may need to use a database like cockroachdb to store the keys, and have an equally scalable application layer that utilises the available blockchain nodes to publish the data.

Comments

Thanks for the answer.

The idea is to rely 100% on the chain (and take full advantage of the 'automatic sync' among nodes). I do not want/wish to rely on classic structures/DB. Getting a solution with a mix of heterogeneous kinds of 'storage' will/may impact the scalability side of a project. If everything is stored in the chain, deploying a new node, closing one, having some kind of load balancing among nodes etc... seems quite straight forward to me : install a new node on some host with 'initprivkey' and 'autosubscribe', put the application-specific layer on top of this, and I know that some time later the new node, once sync'ed, can act as any other node already in place. One can switch off a node for chain backup purpose without any glitch on the overall mech. of the project.

Assuming that all nodes are deployed in controlled environment/hosts (deployment not publicly open), why would it be more dangerous to store private keys in a stream than in some distributed DB?

Answer 2

You could use a stream to broadcast private keys, but then it means that any node on the blockchain can see those private keys.

You might also consider publishing private keys more confidentially, using this sort of technique:

https://www.multichain.com/developers/stream-confidentiality/

And finally, you could just have each node using its own key, and grant write permissions to the stream from all the relevant addresses.

Comments

Thanks for the answer.

1 - By continuing the digging, I discovered (effectively - as I saw some posts about this topic) that 'importprivkey' may take ages when asking for a rescan. Without a rescan, balances are not up to date for the imported address. It seems also that rescanning also blocks all RPC requests, somehow in 'a blind way'. While understanding why it may behave this way, is there any chance to clearly disable RPC on the node that is engaged in a rescan (making the node unavailable for RPC while still effectively running/rescanning), or may be having the ability to perform the rescan from an external process while keeping RPC up and responding for that specific node?

2 - about the last suggestion, the (my) idea was to be able to do actions on behalf the real stream's owner (...use the real owner's address, through 'publishfrom' function). If I understand correctly your suggestion, it would work, but action will be recorded as being performed by the local node's address. While being a possibility, I feel like here loosing an information.

Thanks.

'Best!

---

1 - Asynchronous rescanning is on our roadmap, but not implemented yet. For now your best workaround may be to run two nodes in parallel, and while one is rescanning, you can continue to use RPC on the other.

2 - If publishing from someone else's address, the action is recorded as being performed by that address only, so I think you'll get what you want.