Mining stop - possible bug?

Question

How to replicate: using a single node

- create a new chain
- create a new stream
- subscribe to it
- publish something

When querying for the asset published, we can see that the item was confirmed

- revoke mining rights (e.g. no more miners)
- grant back mining rights

At this point, no more confirmations will take place. We can publish something and see that the confirmation will remain at 0. The previous item's confirmation number will either remain stable.

It seems that mining stopped when number of miners reached 0 and did not restart - is there a way to force a restart ? Is this a bug?

Answer 1

The set of permitted miners for a block is defined by the state of the blockchain at the end of the previous block. So if you have a block in which the last miner is revoked, then there is no way for any new blocks to be added without rolling back the chain. This is (intentionally) a little tricky but can be achieved by doing this on each node:

pause mining,incoming
setlastblock [hash or height of block before the one where the miner was revoked]
clearmempool
resume mining,incoming

Make sure you have run clearmempool on every node before running the resume command on any node, otherwise the mining revocation transaction can be redistributed again over the peer-to-peer network. Note that this will lose all transactions from after the block you rewound to, not just the one that caused the problem. If you need to be more selective, use the command below instead of clearmempool on every node:

setruntimeparam bantx [the txid in which miner was revoked]

Comments

I ran this, seems promising went to height of 11193 straight away, I then send 0 to an address and now I have 11195 on all nodes.

---

I take it, the fork could not be resolved and we went ahead of it? Or what would the explanation be for that block?

---

Great, it looks like the problem is finally resolved!

The last bit was related to the fact that 1aXH had the chain longer than our "correct" branch. Since it was longer, 1aXH had no reason to switch to the shorter branch. Other nodes, on the other hand didn't switch to that bad long branch because we probably banned it in some way couple of weeks ago. Then we made block on the longer chain invalid - very intrusive and non-standard action, in most cases node switches to another (even shorter) branch automatically, but not in this case (I will not bother you with details, it is also related to the wrong sequence of operations we made). We just had to make the switch explicitly (with the last  setlastblock).

But the source of the problem is mining-diversity=1 (it is also related to the question you asked in another thread). If it would be, say, 0.6, simple revoke on one of the admin nodes would solve the problem - you are absolutely right, this is how it should work (but not for mining-diversity=1). Even more, you would not be aware of the problem at all - 3 nodes would continue mining without waiting for non-existent 1C8A.

But when you granted mining permission to 1C8A, with mining-diversity=1 you said "this address should mine every 4th block". But it is the whole point of blockchain - it is immutable ledger - you cannot change any fact stored on blockchain without agreement from the majority of the miners, and with mining-diversity=1 "majority" means ALL miners, including broken 1C8A (which never agree).

That's why the recovery procedure is so complicated - it is against the nature of blockchain, it cannot be simple. And, of course, it requires agreement from majority of the miners (ALL in our case) and these miners should be perfectly synchronized  to do this properly.  

There is no problem with granting/revoking mine permission, but with mining-diversity=1 you should make sure you have private key of the miner you grant permission.  BTW, it is also true for existing miners (1Y, 1A and 1E) - if you lose one of these keys  - you will be stuck again. The simplest way to do this is to back up the wallet (https://www.multichain.com/developers/backing-up-restoring-nodes/)

---

Thanks for your help. Much appreciated. I did not understand mining-diversity=1 was as strict as it is when I did this back then. thanks again.

---

Unfortunately now its not returning any stream items.

https://www.multichain.com/qa/32542/streams-have-no-stream-items