Create SmartFilters from file due to ARG_MAX

Question

So,

I need to create a Transaction Filter which is enormous () due to a few JSON's. There's simply no way to insert that on the terminal due to "getconf ARG_MAX" (the max size of a terminal command) generally being ~250k bytes.

Already tried to put everything on the same line without spaces, but I'm way pass that.

Can you help me in any way? 

Answer 1

Hi,

Try using a JsonRPC wrapper (for example Savoir). That way you can pass your smart filters code as function parameters.

You can find more on Savoir in this link: https://pypi.org/project/Savoir/

Comments

Great suggestion mirabdull, it didn't work, tho.
Using the NodeJS wrapper I got the error:   
{ code: -8,
 message: 'Invalid custom fields or filter name, too long' }  

And the same error code, albeit with a different message, using Savior:
Error! Status code: 500
Text: {"result":null,"error":{"code":-8,"message":"Couldn't compile filter code: SyntaxError: Unexpected identifier"},"id":1}  

I'm having to escape a lot of characters in Javascript, but I removed them to paste it on the python file. I'm trying to use """function ..... """ on Python, but unsure if I should escape something for it.

---

Please follow this syntax for Savoir:
api.create('txfilter', 'filtername', {} ,'function filtertransaction() { var tx=getfiltertransaction(); if (tx.create) return "Stream creation temporarily disabled"; }')

For more on Smart Filters visit: https://www.multichain.com/developers/smart-filters/ (Under Smart Filter Tutorial Section)

---

I'm calling it like that, but as my function spawns across multiple lines and has single quotes I tried to use """string""" in order to not need to escape all the single quotes.

I'm not a heavy python user tho, so if you think there's a significant difference between """x""" and 'x' that might be creating this situation let me know and I'll find a way to work with it!

---

"""string""" should work fine. For example:

stcode = """function filtertransaction() { var tx=getfiltertransaction(); if (tx.create) return "Stream creation temporarily disabled"; }"""

api.create('txfilter', 'filtername1', {} ,stcode)

PS: I'm sorry if I don't understand your question correctly.

---

Basically: either there's a hidden limit for the js-code that we don't know or I'm doing something wrong.

When using """ in Python, do I need to escape any character? For example, in Javascript I might have to do: 'driver'\'s license'

---

No, you don't have to escape anything. Try running a snippet of code to make sure there's no hidden limit.

---

Code in Python (3, to use with Python2 just remove .Savior from the first line) without the "ceps" and "municipios" objects (+1mi lines):
https://gist.github.com/tloriato/8aed7a8385ac7d0d1a6e75d48bfeb4e8

I'm still getting:

Error! Status code: 500
Text: {"result":null,"error":{"code":-8,"message":"Couldn't compile filter code: SyntaxError: Unexpected identifier"},"id":1}

I don't have a clue about why is that, it should have worked I think

---

Are you not using python3?

Please make sure your code's syntax is correct. it seems like you have an extra "}" in your code.

---

I'm using Python 3, yes. Where do you think there's an extra "} "? I just parsed it and it appears correct.

Answer 2

You could try using the MultiChain Web Demo, which provides a web interface for many APIs, including compiling, testing, creating and approving transaction filters.

Comments

Posted here also: https://github.com/MultiChain/multichain/issues/99

---

The likely issue is that you are coming up against one of the size limits for transactions or in-transaction data (since a filter's code is placed inside a transaction output). See the max-std-op-return-size and max-std-tx-size blockchain parameters. You can start a new chain with higher limits, or if that's not an option, use the upgrade mechanism to increase them on the already running chain.

---

So, I realized that I could read the JS directly from the file with Python and use Savior to call the RPC with it, but I'm still getting the same error -8,  'Invalid custom fields or filter name, too long'. However, the file size now is pretty modest ~0.6mb.  


api = Savoir(rpcuser, rpcpasswd, rpchost, rpcport, chainname)

with open("deployReady.js", 'r') as js_file:
    js = js_file.read()
    print(api.create('txfilter', 'smartFilter1', {}, js))


The deployReady is here: https://gist.githubusercontent.com/tloriato/d3aa5df63c12cfec05c9b55f39f7fcfc/raw/51e16c97bc57aa88eb8a77aa2ffc17df1c85dbd1/file.js

Honestly, I'm clueless at this point

---

I have asked the developer team and will get back to you.

---

Thank you, I'll be waiting!

---

So... what I wrote above was incorrect and based on a wrong assumption - my apologies. In the current protocol there's a fixed limit of just under 64 kb of code. I can see how that's a problem in your case, and we're looking into raising this a new protocol version that will be available in the next maintenance release of MultiChain. Apologies in the meantime.

---

Thank you for the support! If it turns out to be a big trouble and you can't make it, feel free to point me in the direction and I'll try to do it myself!