Streams have no stream items

Question

After my upgrade from 1 to 2.1.2, I have no stream items in any of my streams.

 

{"method":"liststreamitems","params":["JackBritain"],"id":"23499862-1621867726","chain_name":"clicktochain"}

[
]

They are all reading as empty.

   "indexes" : {
            "items" : true,
            "keys" : true,
            "publishers" : true,
            "items-local" : true,
            "keys-local" : true,
            "publishers-local" : true
        },
        "synchronized" : true,
        "items" : 0,
        "confirmed" : 0,
        "keys" : 0,
        "publishers" : 0
    },
    {
        "name" : "DNG",
        "createtxid" : "533be2a23f8b61325c991b66405d94e1442acde72e121106923460a8993577e2",
        "streamref" : "2885-267-15187",
        "restrict" : {
            "write" : true,
            "read" : false
        },
        "details" : {
            "Token" : "CA-ca51710111516ec63af15"
        },
        "subscribed" : true,
        "retrieve" : true,
        "indexes" : {
            "items" : true,
            "keys" : true,
            "publishers" : true,
            "items-local" : true,
            "keys-local" : true,
            "publishers-local" : true
        },
        "synchronized" : true,
        "items" : 0,
        "confirmed" : 0,
        "keys" : 0,
        "publishers" : 0
    },

Answer 1

As a first step, can you please try stopping your node than re-running it with the -rescan runtime parameter?

Comments

Unfortunately, it hasn't helped. Anything else?

---

Can you please check that you don't see error messages in
debug.log and wallet/txs.log

If everything is OK, please try to unsubsribe from this stream and subscribe again
(and check errors if this doesn't help)

---

No errors or anything in log, and unscubscribe/subscribe did not help.

---

Could the rescan option on subscribe help?

---

* Could the rescan option on subscribe help?

rescan option is True by default
 
Let's try this:

unsubscribe
liststreams -> make sure  "subscribed" : false
subscribe
liststreams -> make sure  "subscribed" : true,"synchronized" : true

Then, let's try
unsubscribe
liststreams -> make sure  "subscribed" : false
restart with -rescan
subscribe
liststreams -> make sure  "subscribed" : true,"synchronized" : true

If this doesn't help, you can run MultiChain 1.0 again and see if -rescan helps

---

I did the process for one of the streams but to no avail.

[
    {
        "name" : "JackBritain",
        "createtxid" : "1a65d4f3c851605bddd955025c5fa9673692163293d78892d2e4122a47e07a90",
        "streamref" : "52-519-25882",
        "restrict" : {
            "write" : true,
            "read" : false
        },
        "details" : {
            "Token" : "CA-2d51610091519ec4af14"
        },
        "subscribed" : true,
        "retrieve" : true,
        "indexes" : {
            "items" : true,
            "keys" : true,
            "publishers" : true,
            "items-local" : true,
            "keys-local" : true,
            "publishers-local" : true
        },
        "synchronized" : true,
        "items" : 0,
        "confirmed" : 0,
        "keys" : 0,
        "publishers" : 0
    }
]

---

I am running 1.0 with rescan, will let you know shortly

---

Running 1.0 dies with the following error

Please restart multichaind with reindex=1.
 Aborted block database rebuild. Exiting.

---

Do you see any other message (probably in debug.log) why it requires reindex?

---

There are a couple of errors

ERROR: CheckProofOfWork() : nBits below minimum work

ERROR: LoadBlockIndex() : CheckProofOfWork failed: CBlockIndex(pprev=0x49db530, nHeight=5587, merkle=e35d619e9445c37fc31cce36c73dcc14d78d7ca06aef26c51a5c9df200000003, hashBlock=7090618093baf4e01c0051ec24817100d4babe87958d52037b2e97088a8f05fe)

---

Are you saying 2.0 is fine with it and 1.0 requires reindex?

If yes, providing your chain uses round-robin mining, it is very strange error and it looks like a bug in 1.0. Please confirm 2.0 doesn't require reindex with this error

---

Yes 2.0 did not require a reindex, only version 1.0

However, I have done the reindex and busy rescanning now.

---

Still no streamitems. :(

---

Let's try to find them.

Please take some stream you are sure there were stream items and run liststreams <stream-name>, then take first, say, 10 symbols of "createtxid" (2d484192c953, for example) and run in terminal:

cat wallet/txs.log | grep "2d484192c953"

also, please take tx which you are sure had an item in that stream and look for its txid:

cat wallet/txs.log | grep <txid>

---

I have written items to streams without error since the chain resumed mining, but they are not visible either.

There has been no error and I have done the following
- creating stream
- setting permission
- writing to the stream

The new items and the old items are not visible.

---

I see a lot of this

Import: 2, Transferring entity (00000105, 000000001a65d4f3c851605bddd955025c5fa967) to the chain, txs: 0

Everything in the log says txs: 0

It also has this numerous times
CompleteImport: Successfully updated chain after import 1, *removing old records*

this is a snippet of recent activity

2021-05-25 13:58:23    Entity (00000103, 468e3595176c18adb453d73c7270791f527b76b4) added successfully
2021-05-25 13:58:23    Entity (00000203, 468e3595176c18adb453d73c7270791f527b76b4) added successfully
2021-05-25 13:58:26    Entity (00000103, deae4ffbcde8ad4f218191df88bf91712e54d695) added successfully
2021-05-25 13:58:26    Entity (00000203, deae4ffbcde8ad4f218191df88bf91712e54d695) added successfully
2021-05-25 13:58:27    Entity (00000103, 1f4fcd1c52f308bfbb1b2df071335d78166985a2) added successfully
2021-05-25 13:58:27    Entity (00000203, 1f4fcd1c52f308bfbb1b2df071335d78166985a2) added successfully
2021-05-25 13:58:29    Entity (00000103, fc65177f5c808de39262d16325e0150d04895cab) added successfully
2021-05-25 13:58:29    Entity (00000203, fc65177f5c808de39262d16325e0150d04895cab) added successfully
2021-05-25 13:58:31    NewTx 695278ccce681c4a5cca1aca20282fdc62777442b6cdaa70e01bf6cb21a4090d, block -1, flags 00030004, import 0
2021-05-25 13:58:33    NewTx c060bd581ea14e1d4c6c301952712027b71f7f48570d508515bfd0ff34291c03, block -1, flags 00030004, import 0
2021-05-25 13:58:34    NewTx d16ecd0165541da78c361a657f54d28de41f288ee65443e700681a5b0fc3f4bc, block -1, flags 00030004, import 0
2021-05-25 13:58:36    NewTx f248a3bb3691ea6bc10bed43cc45d96f5b8d6af55ef007c6bdb4817d4fbb2f1c, block -1, flags 00030004, import 0
2021-05-25 13:58:37    NewTx 5b163ab5e0e1654e316d1142834f8bc50b9302e782929a69e51b753f30834750, block -1, flags 00030004, import 0
2021-05-25 13:58:38    NewTx ab970d3e52943bb961b8eaa7211f0d8ea8d8f313ae4ed0076d438e343c8a0e66, block -1, flags 00030004, import 0
2021-05-25 13:58:39    NewTx f5cfa94c2616dc3cdaffbe31fff377ccee79700eda5aa963170e4190b407737d, block -1, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx 695278ccce, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx c060bd581e, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx d16ecd0165, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx f248a3bb36, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx 5b163ab5e0, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx ab970d3e52, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    UpdateTx f5cfa94c26, block 11246, flags 00030004, import 0
2021-05-25 13:58:48    NewBlock 11246, Txs: 7, Rows: 56
2021-05-25 13:58:48    NewTx 1ca6cd48468f18fb97d02b4650cf3900a0c4aad39a39d0a74cc9feeaa585205a, block -1, flags 00030004, import 0
2021-05-25 13:58:56    UpdateTx 1ca6cd4846, block 11247, flags 00030004, import 0
2021-05-25 13:58:56    NewBlock 11247, Txs: 1, Rows: 6
2021-05-25 13:59:12    NewBlock 11248, Txs: 0, Rows: 0

---

[
    {
        "name" : "SAPX",
        "createtxid" : "c1f4579ea25a051981f40688a1858f17a52fb51074e29858c485db44f1680ca5",
        "streamref" : "11202-811-62657",
        "restrict" : {
            "write" : true,
            "read" : false
        },
        "details" : {
            "Token" : "CA-ad52110051507ec118af16"
        },
        "subscribed" : true,
        "retrieve" : true,
        "indexes" : {
            "items" : true,
            "keys" : true,
            "publishers" : true,
            "items-local" : true,
            "keys-local" : true,
            "publishers-local" : true
        },
        "synchronized" : true,
        "items" : 0,
        "confirmed" : 0,
        "keys" : 0,
        "publishers" : 0
    }
]

liststreamitems
{"method":"liststreamitems","params":["SAPX"],"id":"28724021-1621948925","chain_name":"clicktochain"}

[
]

This is the create stream tx and entities
2021-05-25 14:19:52    Entity (00000105, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Entity (00000205, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Entity (00000106, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Entity (00000206, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Entity (00000107, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Entity (00000207, 00000000c1f4579ea25a051981f40688a1858f17) added successfully
2021-05-25 14:19:52    Starting new import 3 with 6 entities from block 11201

---

And what happens in the log after you publish in this stream?

---

Wait a minute, what do you see after "Starting new import 3 with 6 entities from block 11201"?

---

Starting new import 3 with 6 entities from block 11201
2021-05-25 14:19:52    Import: 3, Entity: (00000105, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import: 3, Entity: (00000205, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import: 3, Entity: (00000106, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import: 3, Entity: (00000206, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import: 3, Entity: (00000107, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import: 3, Entity: (00000207, 00000000c1f4579ea25a051981f40688a1858f17), transferred txs: 0
2021-05-25 14:19:52    Import 3 successfully started
2021-05-25 14:19:52    Tx 10f0f49d84904f0d94e59ee3578e10d6129a1589e83cee68c53ef7169d25e379 ignored for import 3
2021-05-25 14:19:52    Tx e859882133bfcca9d551f45940645b3ef6314a9ecd557feb5c5815bc3d343bc4 ignored for import 3
2021-05-25 14:19:52    Tx c1f4579ea25a051981f40688a1858f17a52fb51074e29858c485db44f1680ca5 ignored for import 3
2021-05-25 14:19:52    NewBlock 11202, Txs: 0, Rows: 0
2021-05-25 14:19:52    NewBlock 11203, Txs: 0, Rows: 0
2021-05-25 14:19:52    Tx 6101e94a24d47480ad7590fb8f6b7c35f72b61b2b9a1321860ccf2b0f7bfcac4 ignored for import 3
2021-05-25 14:19:52    Tx 0ee016680f7c24a76265bdefd27cb29b94c8adf8092a8eaffc7ddec98dee2c70 ignored for import 3
2021-05-25 14:19:52    Tx c2cda87e78c12d8d13857890e9167e7c30b79dbf12f774788afb38a62737e0ac ignored for import 3
2021-05-25 14:19:52    Tx 10f2e7f48916c47528efc4f2f1a5c0c92b92f193fe235ef887b265e32ed153d4 ignored for import 3
2021-05-25 14:19:52    Tx 483cfdc2a3563bd366768e80b1de9f17cea0a02af92e58097330ae43f52aed99 ignored for import 3
2021-05-25 14:19:52    Tx f2ba43219cf6b1734ee492580b65ed12ac1db0d4ab2a7508ca4e651903746613 ignored for import 3


continued...

CompleteImport: Successfully updated chain after import 3, removing old records

---

OK, but what about "Transferring entity ... 00000000c1f4579ea25a051981f40688a1858f17)

before "Successfully updated chain after import 3..."

Do you see 6 records all with txs: 0?

And what do you see about Tx publishing in this stream, is it ignored too?

---

Do you see 6 records all with txs: 0?
Yes

is it ignored too?

Everything says ignored.

I subscribed to another stream and ran liststreamitems, the log shows ignored and txs 0 as well.

---

Can you please send an output of

getrawtransaction <txid> 1

where txid is tx you are sure you published to stream SAPX?

---

I think this is one.
{"method":"getrawtransaction","params":["8eaad1088d34007d8a13c159c9092746514da5e8bbde5d72e2bd7bf8af7e71ce",1],"id":"12502212-1621962256","chain_name":"clicktochain"}

{
    "hex" : "010000000198400692d416eaa2de2310a69dfbcfafcb6c91211e6130449a3ceb62541408cd000000006a47304402207d2975d0e3d53b78ce0a3fc671543d7c4a11cd62b0c0a9618577ac3454d7a31702206f60f1443325a3fd15aea5bb80506650120e2b82b67386ffdcdeb4bef065d846012102d7cfde050748f8d46ea9742a4e98b8878d5078cfd995fdef346d61466867a641ffffffff0300000000000000001976a914a98c32a50b639b6518de1f3e8c3453b39560bf9288ac0000000000000000fd24166a4d20165044393462577767646d567963326c76626a30694d5334774969426c626d4e765a476c755a7a30696458526d4c544532496a382b5045567559334a35634852706232354559585268494868746247357a4f6e687a5a4430696148523063446f764c336433647935334d793576636d63764d6a41774d533959545578545932686c62574569494868746247357a4f6e687a615430696148523063446f764c336433647935334d793576636d63764d6a41774d533959545578545932686c625745746157357a6447467559325569506a7846626d4e79655842305a575245595852684946523563475539496d6830644841364c79393364336375647a4d7562334a6e4c7a49774d4445764d445176654731735a57356a493056735a57316c626e5169494868746247357a50534a6f644852774f693876643364334c6e637a4c6d39795a7938794d4441784c7a41304c3368746247567559794d69506a7846626d4e796558423061573975545756306147396b494546735a3239796158526f625430696148523063446f764c336433647935334d793576636d63764d6a41774d5338774e4339346257786c626d4d6a5957567a4d6a55324c574e69597949674c7a3438533256355357356d6279423462577875637a30696148523063446f764c336433647935334d793576636d63764d6a41774d4338774f5339346257786b63326c6e4979492b5045567559334a356348526c5a45746c6553423462577875637a30696148523063446f764c336433647935334d793576636d63764d6a41774d5338774e4339346257786c626d4d6a496a34385257356a636e6c7764476c76626b316c644768765a43424262476476636d6c3061473039496d6830644841364c79393364336375647a4d7562334a6e4c7a49774d4445764d445176654731735a57356a49334a7a59533078587a55694943382b5045746c65556c755a6d386765473173626e4d39496d6830644841364c79393364336375647a4d7562334a6e4c7a49774d4441764d446b76654731735a484e705a794d69506a78594e5441355247463059543438574455774f554e6c636e52705a6d6c6a5958526c506b314a53555a61616b4e44516b55325a30463353554a425a306c5257444a50556a5647646d3077543235615a586445626e49346257647a656b464f516d64726357687261556335647a42435155...",
    "txid" : "8eaad1088d34007d8a13c159c9092746514da5e8bbde5d72e2bd7bf8af7e71ce",
    "version" : 1,
    "locktime" : 0,
    "vin" : [
        {
            "txid" : "cd08145462eb3c9a4430611e21916ccbafcffb9da61023dea2ea16d492064098",
            "vout" : 0,
            "scriptSig" : {
                "asm" : "304402207d2975d0e3d53b78ce0a3fc671543d7c4a11cd62b0c0a9618577ac3454d7a31702206f60f1443325a3fd15aea5bb80506650120e2b82b67386ffdcdeb4bef065d84601 02d7cfde050748f8d46ea9742a4e98b8878d5078cfd995fdef346d61466867a641",
                "hex" : "47304402207d2975d0e3d53b78ce0a3fc671543d7c4a11cd62b0c0a9618577ac3454d7a31702206f60f1443325a3fd15aea5bb80506650120e2b82b67386ffdcdeb4bef065d846012102d7cfde050748f8d46ea9742a4e98b8878d5078cfd995fdef346d61466867a641"
            },
            "sequence" : 4294967295
        }
    ],
    "vout" : [
        {
            "value" : 0,
            "n" : 0,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 a98c32a50b639b6518de1f3e8c3453b39560bf92 OP_EQUALVERIFY OP_CHECKSIG",
                "hex" : "76a914a98c32a50b639b6518de1f3e8c3453b39560bf9288ac",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1Pv5doo7ngC8LSpqabapwFWjgxQGpYKmQQtAXF"
                ]
            }
        },
        {
            "value" : 0,
            "n" : 1,
            "scriptPubKey" : {
                "asm" : "OP_RETURN 5044393462577767646d567963326c76626a30694d5334774969426c626d4e765a476c755a7a30696458526d4c544532496a382b5045567559334a35634852706232354559585268494868746247357a4f6e687a5a4430696148523063446f764c336433647935334d793576636d63764d6a41774d533959545578545932686c62574569494868746247357a4f6e687a615430696148523063446f764c336433647935334d793576636d63764d6a41774d533959545578545932686c625745746157357a6447467559325569506a7846626d4e79655842305a575245595852684946523563475539496d6830644841364c79393364336375647a4d7562334a6e4c7a49774d4445764d445176654731735a57356a493056735a57316c626e5169494868746247357a50534a6f644852774f693876643364334c6e637a4c6d39795a7938794d4441784c7a41304c3368746247567559794d69506a7846626d4e796558423061573975545756306147396b494546735a3239796158526f625430696148523063446f764c336433647935334d793576636d63764d6a41774d5338774e4339346257786c626d4d6a5957567a4d6a55324c574e69597949674c7a3438533256355357356d6279423462577875637a30696148523063446f764c336433647935334d793576636d63764d6a41774d4338774f5339346257786b63326c6e4979492b5045567559334a356348526c5a45746c6553423462577875637...",
                "type" : "nulldata"
            },
            "data" : [

---

This transaction doesn't look like "publish-to-stream".  How did you create it?

---

I couldnt locate a txid for some reason, so I just ran a new publishfrom
Then I listed it and its there in the stream. The question is what happened to everything that has been added for the last 4 years?

{"method":"getrawtransaction","params":["d5ee913c156d13d3e6129db08929dfc3ca48e3ef97e00a2751947787a285ea61",1],"id":"52919400-1621973594","chain_name":"clicktochain"}

{
    "hex" : "010000000111a99ab954adf5f18a253df74461eb0449f67eeb18226432c0b9ed8e062119d3020000006a47304402202403a5351032f9ad02422c1d04b24f56f817711a381a98073c4aa929b2b04e9002204a1d3e0be56789c15256c91111b8479a5b85aef4679ba9510be7daf8f25481df0121028c2386ecbf223a771142b938442ab8cf9c1085baece3374b35b40ee8adbe3240ffffffff0200000000000000003b1473706b6567a95f5c0255d9dd5b6051c8f3d4651a750b73706b6b546573744b6579756a167b2274657874223a2268656c6c6f20776f726c64227d00000000000000001976a914d46c09025bf74c2c0d66d8e1fa9d84023289a5fa88ac00000000",
    "txid" : "d5ee913c156d13d3e6129db08929dfc3ca48e3ef97e00a2751947787a285ea61",
    "version" : 1,
    "locktime" : 0,
    "vin" : [
        {
            "txid" : "d31921068eedb9c032642218eb7ef64904eb6144f73d258af1f5ad54b99aa911",
            "vout" : 2,
            "scriptSig" : {
                "asm" : "304402202403a5351032f9ad02422c1d04b24f56f817711a381a98073c4aa929b2b04e9002204a1d3e0be56789c15256c91111b8479a5b85aef4679ba9510be7daf8f25481df01 028c2386ecbf223a771142b938442ab8cf9c1085baece3374b35b40ee8adbe3240",
                "hex" : "47304402202403a5351032f9ad02422c1d04b24f56f817711a381a98073c4aa929b2b04e9002204a1d3e0be56789c15256c91111b8479a5b85aef4679ba9510be7daf8f25481df0121028c2386ecbf223a771142b938442ab8cf9c1085baece3374b35b40ee8adbe3240"
            },
            "sequence" : 4294967295
        }
    ],
    "vout" : [
        {
            "value" : 0,
            "n" : 0,
            "scriptPubKey" : {
                "asm" : "73706b6567a95f5c0255d9dd5b6051c8f3d4651a OP_DROP 73706b6b546573744b6579 OP_DROP OP_RETURN 7b2274657874223a2268656c6c6f20776f726c64227d",
                "hex" : "1473706b6567a95f5c0255d9dd5b6051c8f3d4651a750b73706b6b546573744b6579756a167b2274657874223a2268656c6c6f20776f726c64227d",
                "type" : "nulldata"
            },
            "items" : [
                {
                    "type" : "stream",
                    "name" : "JackBritain",
                    "createtxid" : "1a65d4f3c851605bddd955025c5fa9673692163293d78892d2e4122a47e07a90",
                    "streamref" : "52-519-25882",
                    "publishers" : [
                        "1ViAvHsMveEFMvUN4aRmoPMoxitR8sNr3SPJ6G"
                    ],
                    "keys" : [
                        "TestKey"
                    ],
                    "offchain" : false,
                    "data" : "7b2274657874223a2268656c6c6f20776f726c64227d"
                }
            ]
        },
        {
            "value" : 0,
            "n" : 1,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 d46c09025bf74c2c0d66d8e1fa9d84023289a5fa OP_EQUALVERIFY OP_CHECKSIG",
                "hex" : "76a914d46c09025bf74c2c0d66d8e1fa9d84023289a5fa88ac",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1ViAvHsMveEFMvUN4aRmoPMoxitR8sNr3SPJ6G"
                ]
            }
        }
    ]
}

---

* The question is what happened to everything that has been added for the last 4 years?

I cannot say without seeing the transaction.

Normal publish transaction should have on one of the outputs

73706b65... OP_DROP 73706b6b... OP_DROP OP_RETURN ...

(like in tx you just sent). If you find one like this we'll try to understand what's going one

I tried to simulate this behavior and upgraded from 1.0 to 2.1.2 with rescan - I see no problem

---

Sorry, not enough time in a day :)

{
    "hex" : "010000000152aeca4bce066b4d6f84c66d5ae43c513a98ec5928a891a5846bd7ba0b04f8b1020000006b483045022100ff916bfe3d8d105ded1fff750c52cf4d5ee49cd892a0e8e5993e0d40ef382674022023c679494c265db8ddcf8d4b37ce3e74eec61f1627ebbf8354456a073fdd69c9012102011158f7adf0a5a19414d087e55f5246230db0db25b883c7009bc40b5ecf8a12ffffffff0300000000000000002776a914a98c32a50b639b6518de1f3e8c3453b39560bf9288ac0c73706b670100000000000000750000000000000000534c4f73706b6e0100011050412d37616563326472313761663130004104010000005075626c69736865644461746500114e6f20446174652053706563696669656450726963650009342c3733302c303030756a00000000000000001976a914eacdaa7df376714f46032105fc565d3e12bfbb1288ac00000000",
    "txid" : "7c6a1a5aa4c76bac6b439e31b1ce5215d2140c0ed6cfc32006e42b687eef592c",
    "version" : 1,
    "locktime" : 0,
    "vin" : [
        {
            "txid" : "b1f8040bbad76b84a591a82859ec983a513ce45a6dc6846f4d6b06ce4bcaae52",
            "vout" : 2,
            "scriptSig" : {
                "asm" : "3045022100ff916bfe3d8d105ded1fff750c52cf4d5ee49cd892a0e8e5993e0d40ef382674022023c679494c265db8ddcf8d4b37ce3e74eec61f1627ebbf8354456a073fdd69c901 02011158f7adf0a5a19414d087e55f5246230db0db25b883c7009bc40b5ecf8a12",
                "hex" : "483045022100ff916bfe3d8d105ded1fff750c52cf4d5ee49cd892a0e8e5993e0d40ef382674022023c679494c265db8ddcf8d4b37ce3e74eec61f1627ebbf8354456a073fdd69c9012102011158f7adf0a5a19414d087e55f5246230db0db25b883c7009bc40b5ecf8a12"
            },
            "sequence" : 4294967295
        }
    ],
    "vout" : [
        {
            "value" : 0,
            "n" : 0,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 a98c32a50b639b6518de1f3e8c3453b39560bf92 OP_EQUALVERIFY OP_CHECKSIG 73706b670100000000000000 OP_DROP",
                "hex" : "76a914a98c32a50b639b6518de1f3e8c3453b39560bf9288ac0c73706b67010000000000000075",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1Pv5doo7ngC8LSpqabapwFWjgxQGpYKmQQtAXF"
                ]
            },
            "assets" : [
                {
                    "name" : "PA-7aec2dr17af10",
                    "issuetxid" : "7c6a1a5aa4c76bac6b439e31b1ce5215d2140c0ed6cfc32006e42b687eef592c",
                    "assetref" : "139-266-27260",
                    "qty" : 1,
                    "raw" : 1,
                    "type" : "issuefirst"
                }
            ]
        },
        {
            "value" : 0,
            "n" : 1,
            "scriptPubKey" : {
                "asm" : "73706b6e0100011050412d37616563326472313761663130004104010000005075626c69736865644461746500114e6f20446174652053706563696669656450726963650009342c3733302c303030 OP_DROP OP_RETURN",
                "hex" : "4c4f73706b6e0100011050412d37616563326472313761663130004104010000005075626c69736865644461746500114e6f20446174652053706563696669656450726963650009342c3733302c303030756a",
                "type" : "nulldata"
            }
        },
        {
            "value" : 0,
            "n" : 2,
            "scriptPubKey" : {
                "asm" : "OP_DUP OP_HASH160 eacdaa7df376714f46032105fc565d3e12bfbb12 OP_EQUALVERIFY OP_CHECKSIG",
                "hex" : "76a914eacdaa7df376714f46032105fc565d3e12bfbb1288ac",
                "reqSigs" : 1,
                "type" : "pubkeyhash",
                "addresses" : [
                    "1Yjcr4p4qex1vpV9QDpuomNocGxu45EM9RQUoD"
                ]
            }
        }
    ],
    "issue" : {
        "type" : "issuefirst",
        "name" : "PA-7aec2dr17af10",
        "multiple" : 1,
        "open" : false,
        "details" : {
            "PublishedDate" : "No Date Specified",
            "Price" : "4,730,000"
        }
    },
    "blockhash" : "00b4d130194ea2f8519c3633051274ff69b4ae7fa14b503396d4ffd903f9c5e3",
    "confirmations" : 11156,
    "time" : 1506063859,
    "blocktime" : 1506063859
}

---

Sorry, it is not stream item either. It is asset issuance transaction

---

I have searched dozens and dozens manually. How would I find this without liststreamitems?

---

I don't see easy way, but you can try to find tx with OP_DROP 73706b6b

multichain-cli <chain-name> getblock <block> 4 | grep "OP_DROP 73706b6b"

and write some script running <block> in loop. If you find a block, you can check its txs, one by one

---

Im running a loop - so far height of 2300 - not sign of OP_DROP 73706b6b whatsoever. I am getting worried now.
Is the data still there? can we recover this?

---

The data cannot disappear if you successfully made -reindex - it scans and rechecks all data. You may have a fork, but the fork with length 12K+ blocks is very unlikely, especially if you see other data you had (like asset issuances).

So, if the data was there, it is there, somewhere in transactions inside blocks. You cannot lose or change a single transaction confirmed in block - it is entire point of blockchain

Is there any chance that your application used some other mechanism (not streams) for storing the data. For example, like in tx you sent before - it has large blob of raw data, but not stream item.

---

Perhaps you have some application logs which recorded when this stream data was supposed to have been written, and you can compare that against block timestamps (give or take a few minutes in either direction) to find the appropriate block in which it should be located. And is it possible you created more than one blockchain over the years, and the data was written in a different one?

---

I have done that as well, 10 blocks either side of a transaction should have provided me with stream data. I have checked 10 of these cases.

I have also now checked 3400 blocks, with no result.

This has been running for 4 and half years on this chain, with the same code, no issues until mining stopped and we went to version 2.1.2.

---

Can you please clarify what do you mean by "10 blocks either side of a transaction should have provided me with stream data"?

What is the "transaction" and how exactly this "stream data" is created?

---

Is it possible you were using this "quasi-streams" method, before MultiChain's proper streams were available? The transaction you posted looks more like that.

https://www.multichain.com/developers/quasi-data-streams/

---

I create a stream, grant permissions, create wallets for people early on.

When actions occur on the system, I am writing to the stream.
I am using publishfrom

A user is led through a workflow
(simplied)
fill in a verification details
credit check
make offer
2-Factor Authentication
accept offer
reject offer

I write data to the stream when these actions occur. I have an audit trail in my database and the timestamps, which can help me look at blocks and transactions around the same time.

So I looked at an audit item's timestamp and went to the chain looking for the equivalent time and then went 10 blocks either side of that. Maybe I am wrong here, but I thought that I am broadening my search for OP_DROP 73706b6b.

If I found the transaction on block 1000, then searched through blocks 900 - 1100 to find OP_DROP.

I am not sure how else to find OP_DROP

---

I am not sure what quasi streams was before your comment. I am using

publishfrom address stream key data

Is this different?

---

If you used publishfrom, this is regular streams. And you know the publishing was successful previously because publishfrom returned a txid, and/or because you could retrieve the items using APIs like liststreamkeyitems?

---

Yes, absolutely. it's been running perfectly well for years.
The explorer here was showing the stream data as well, but now its not finding any.

https://clicktopurchase.com/BlockChain/Explorer/

This transactions here

https://clicktopurchase.com/BlockChain/Explorer/Tx/5c0dbff66bdd55101377823d7c557d0ab8c1e86cbdf6f9a60f87822015122b1a

Add additional data pulled through from a stream, so we were seeing this all the time. I will find the exact call we used to retrieve but I think so,

The point is that the data was there, and now its not.
What has changed with streams since v1?
We serialize an object into the transaction - has the size of data changed perhaps? Anything on keys (names, formats, length)?

---

What will it take to get you to help me sort this out more directly?

---

There is a maxshowndata runtime parameter that could be relevant for data retrieval but it wouldn't affect the number of items shown for a stream.

Did you test what happens if you publish a stream item now, that the transaction appears in the chain and a stream item is shown in liststreams/liststreamitems?

In terms of sorting this out directly, we can take a look at the blockchain directory from one of the nodes, but the question is whether you are comfortable with us doing this.

---

I did a publishfrom the commandline and added something - I could list it again.

i am perfectly comfortable for you to help out, if possible, please send me direct message and we can arrange?

---

OK so please stop the node, zip up the blockchain directory, and send it to multichain dot debug at gmail dot com. We'll reply from there.

---

We've received your chain, you may remove the link.

I don't see any stream items on this chain (except the test item you published recently)

I see lot of data transactions containing encrypted XMLs. Each asset transfer contains also  some XML metadata. Is this expected? May this be somehow related to the stream items?

I think the only thing we can do now is to wait for the next "publish" event and see whether you see stream item.

If you think the problem is with 2.0 (unlikely in my opinion) I can suggest you to start new node with 1.0, grant to its address only connect permission, wait for the node to synchronize with the network and see how this "publish" event  appears on this node

Several questions which may help

1. Please confirm that you have only 3 nodes, all with one address with mining permission

2. What was the size of these missing items. KBs, MBs?

3. What is the version of 1.0 you were using?

4. Do you have a backup the node which published these items?

5. Any chance you are using multichain-cli in background to publish these items?

---

Your questions are making me question my sanity. It was 4 years ago. I plan to go back into the code and verify the flow and get my picture of it more accurate again.

What I remember
- create stream
- publishfrom into stream

We serialize an object we call chaindata. In it are fields that we deemed necceary to store for an audit like system.

I have 3 nodes - never more.
I have only granted access to one address per node - I have no idea why all addresses are reporting as local to the one node.

The file version is 0.10.0.0 with multichaind reporting
MultiChain 1.0.1 Daemon (protocol 10004-10009)
Multichain-cli has a file version of 0.10.0.0 and reporting MultiChain 1.0.1 RPC client.

I call the rpc with LucidOcean.Multichain c# library.
https://github.com/JonathanCrossland/multichain/blob/master/LucidOcean.MultiChain/API/Stream.cs

In that file, the code calls Stream.Create with .write permissions, it uses publishfrom to publish XML serialization from ChainData Type (in another codebase).

The size of data is variable as it contains mainly strings like status, name,  address.

I am not sure what you mean by using multichain-cli in the background. If you mean do I spawn an instance of the cli and run commands - then no. I talk directly to the rpc server via c#.

I have a backup, only in the sense of a single zip of the chaindata folder.

Perhaps the data is being written directly to the address and not the stream?
I can check, however, should I still not be able to see it now with the same code that has always worked? Has anything changed with regards to data being written in this way?

---

Nothing was changed in the way transactions created by publishfrom are written.

This is what I see in the chain:

1. There is only one transaction created with publish(from) - your latest test transaction

2. There are lot of transactions created with sendwith(meta)data(from) or similar API containing XMLs. These transactions can be associated with specific address using APIs like getaddresstransaction.

So, unless you are sure txs of type 2 are not what are you looking for, I suspect that they can be used for some stream simulation on the application level. publishfrom was never actually called (at least successfully)

So, this is what I suggested: Let's wait for the next "publishfrom" event after (if it not occurred yet) and see if and how it is represented on your chain.

---

thanks for your help. I can verify that what you said seems true (the publishfrom perhaps not working as what the codebase was expecting) A number of us worked on this part of the code, so I need to dive in.

I am going to verify all of my calls and improve logging etc, so I can truly gauge what my side is doing..

---

I sent two files to you. I noticed a difference in the result of  getrawransaction tx 1
I have send 1.0 and 2.1.2 outputs.

---

First, I think you mislabelled the files, 1.0 is 2.1.2 and vice versa.

Anyway, it looks OK, some fields are added in version 2.1.2

Some fields are deprecated in 2.1.2, see

https://www.multichain.com/developers/json-rpc-api/

Maybe what you are missing is top-level "data" field, which in 2.1.2 is returned for each output separately (because in 2.0+, multiple data outputs are allowed). In any case, it has nothing to do with streams,  

To restore all these missing fields you should run 2.1.2 with -v1apicompatible runtime parameter

---

Sorry about the file names :)

Ok, I will get further information regarding my streams - I have code that *is* calling streams, creating them etc - but I see the data I am after wasn't there after all.
This has come as a surprise. I will follow up once I have checked what is going on there.

thanks for your help, I learnt something valuable here, thanks.

---

As a follow up to -v1apicompatible

Would this flag have any consequence on running -rescan or looking at liststreamitems ?

---

No, the only difference is how the keys are represented in liststreamitems (see the link I sent, search for v1apicompatible).

But this will not affect the fact that there are no stream items.