I2PSnark: "InvalidBEncodingException: Infohash mismatch" after changing tracker

[killyourtv]

I2P 0.9.13-4

The torrent was fetched using Azureus from TPB. The tracker was switched using the script at the bottom of this ticket.

After adding the torrent to I2PSnark I got the following exception.

06/09/14 19:53:49.346 ERROR [k DirMonitor] org.klomp.snark.SnarkManager : Torrent in "filename.torrent" is invalid: Infohash mismatch, please report
     org.klomp.snark.bencode.InvalidBEncodingException: Infohash mismatch, please report
     at org.klomp.snark.MetaInfo.<init>(MetaInfo.java:129)
     at org.klomp.snark.MetaInfo.<init>(MetaInfo.java:115)
     at org.klomp.snark.SnarkManager.addTorrent(SnarkManager.java:949)
     at org.klomp.snark.SnarkManager.monitorTorrents(SnarkManager.java:1693)
     at org.klomp.snark.SnarkManager.access$1200(SnarkManager.java:46)
     at org.klomp.snark.SnarkManager$DirMonitor.run(SnarkManager.java:1524)
     at java.lang.Thread.run(Thread.java:744)
     at net.i2p.util.I2PThread.run(I2PThread.java:84)

FWIW, btshowmetainfo torrentfile doesn't report any errors.

I set the tracker to my open tracker for testing. Azureus, Robert, and PyBit? accepted the converted torrent without complaining. Only I2PSnark rejected it.

#!/usr/bin/env python

#import bencode
from BitTornado import bencode
import sys
from os import path

torrent_file = open(sys.argv[1])

#tracker = 'http://tracker2.postman.i2p/announce.php'
tracker = 'http://5mpvzxfbd4rtped3c7ln4ddw52e7i7t56s36ztky4ustxtxrjdpa.b32.i2p/a'
decoded_data = bencode.bdecode(torrent_file.read())

# these 'pop' lines are a cheap hack to remove these entries from the torrent file
# (see https://stackoverflow.com/questions/11277432/how-to-remove-a-key-from-dictionary for more info)
# * announce-list: remove all existing trackers
# * azureus*properties: azureus specific data. Includes peers and tracker data
# * torrent filename: includes the full path of the torrent file
decoded_data.pop('announce-list', None)
decoded_data.pop('azureus_properties', None)
decoded_data.pop('azureus_private_properties', None)
decoded_data.pop('torrent filename', None)
decoded_data['announce'] = tracker

#Writing the torrent file
f = open("new_"+path.basename(sys.argv[1]).replace(' ', '_'), "w")
f.write(bencode.bencode(decoded_data))
f.close()

[zzz]

Snark has some hackery in it where it only sores in memory the parts of the metainfo it cares about.

This wasn't a problem until we added the ut_metadata extension; now it must transfer the info dictionary to peers unchanged, or the infohash won't match. But it uses the in-memory structure rather than pulling from the stored torrent file.

Can't remember for sure, but I think I put that exception in there to catch cases with non-standard entries in the info dict. But I also thought I fixed the problem by storing all entries in a byte array too.

Anyway, this torrent file (supplied by the OP) has three non-standard entries: file-duration, file-media, and profiles, hence the exception.

Will have to figure out if things work, and we can simply remove the test and exception, or if we must do more work to remember all entries. This is more important now that Vuze peers are bridging to clearnet, and there's more incentive to modify clearnet torrents for in-I2P use.

For further research.

[killyourtv]

It looks like µTorrent creates torrents with these non-standard fields, which means that there will be a crapload of these in the wild.

[killyourtv]

ruTorrent had the ​same problem.

[zzz]

Fixed in c6bbe60ecc0413be626f2ec09c12d9a28db85bdc 0.9.13-6.

The problem wasn't at all what I speculated in comment 3 above. The decoding of negative numbers was broken; a -1 was parsed as 11. So when re-encoding to verify the infohash, it failed.

The reference torrent file sent to me by the OP contained:

file-media=BEValueBEValue[0], BEValue[0], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1], BEValue[-1?]

… which triggered the bug. We've just never encountered negative numbers before, apparently.

[zzz]

Additional fix in 6276d8b6eea8401a4c0b66d0033fb86f1235025e 0.9.13-7 for this:

 17:23:17 | Thread terminated unexpectedly: Torrent File EepGet
 17:23:17 | java.lang.NullPointerException
 17:23:17 |  at org.klomp.snark.web.FetchAndAdd.add(FetchAndAdd.java:181)
 17:23:17 |  at org.klomp.snark.web.FetchAndAdd.run(FetchAndAdd.java:97)
 17:23:17 |  at java.lang.Thread.run(Thread.java:724)
 17:23:17 |  at net.i2p.util.I2PThread.run(I2PThread.java:84)