Opened 2 years ago

Closed 13 months ago

#2473 closed enhancement (fixed)

Lower Snark piece size

Reported by: jogger Owned by: zzz
Priority: minor Milestone: 0.9.46
Component: apps/i2psnark Version: 0.9.39
Keywords: Cc:
Parent Tickets: Sensitive: no

Description

As a frequent user of BiglyBT I noticed that choosing lower piece size really helps with slow and unreliable leechers who frequently see transfers of individual pieces aborted. (A 4 MB piece will take > 1 hour to transfer at 1 KBps).

Snarks currently only uses powers of 2 for piece size when creating, but supports any valid multiple of 1 KB when leeching. There seems to be a minimum of 256 KB whereas 16 KB are allowed.

I suggest to choose the smallest possible piece size within the bittorrent spec so that the limits of the most strict tracker are still met which currently is Postman at 20480 pieces.

Subtickets

Change History (6)

comment:1 Changed 2 years ago by zzz

In bittorrent, transfers are done in chunks (typically 16 KB), not pieces. Piece transfers cannot be "aborted", only chunks. The piece size is set in the torrent.

Many (most?) torrents these days are via magnets to torrents bridged from the clearnet by biglybt, not torrents created by snark and put up on postman. We have no control of the piece size for those.

Small piece sizes just make the torrent files bigger. There should not be any performance or reliability difference from changing the piece size.

comment:2 Changed 2 years ago by jogger

There are definitely gains from a smaller piece size. Seeing it every day on my torrents.

Leechers are earlier ready to distribute a piece and if they are slow, probability is increased it is at all usable by someone else.

Those slower leechers come and go frequently. You should see that watching your torrents. Once a connections dies there is no chance of checksumming the piece in transfer and all chunks are discarded. Have seen many leechers in the past that did never make any real progress on Snark-created torrents with 8 MB piece size.

comment:3 Changed 2 years ago by zzz

It's true that in tit-for-tat a leecher is more likely to be unchoked by a peer if the leecher has something to offer. so it will have slower download until it gets the complete first piece. Since it's rarest-first, things usually speed up immediately after that.

It is not true that "all chunks are discarded" when a peer disconnects in snark. Snark saves the partial piece and will prioritize completing it if possible.

"real progress" is happening behind the scenes as the chunks keep coming in. However, the UI only updates the completion info when a complete piece is received.

Piece size does not make a difference, except, perhaps, for a brand new leecher starting from zero, and that resolves itself in seconds to minutes, after the first piece is complete.

comment:4 Changed 2 years ago by jogger

I do not want to start a long argument, I just only believe what I can measure. BiglyBT can be configured for extensive stats. Running dozens of torrents on quite a number of instances I clearly see single snark leechers at 1-2 KiBps? or lower (long time average) lagging behind exactly if piece size > 256 KiB only.

comment:5 Changed 2 years ago by zzz

Not discounting your results, just don't understand them yet. Will investigate further, but it's low priority atm.

comment:6 Changed 13 months ago by zzz

Milestone: undecided0.9.46
Resolution: fixed
Sensitive: unset
Status: newclosed

Added preference to lessen chance that peers with 0-3 pieces will be choked, which will let them become interesting faster and get unchoked by other peers. This addresses my point in last paragraph of comment 3 above. I think this change will not have a big effect on anything except when the peer is low bandwidth and the pieces are huge (perhaps 1 MB or more).

In 14ec2450a6225f031858becd63d4ccafdfc81737 to be 0.9.45-7.

I've been unable in the past year to think of another reason why piece size would matter.

I do see that transfer speeds from bigly to snark are frequently terrible, less than 1 KBps and much worse than snark. It may be a bandwidth allocation or limiting issue for them. I may report this to the bigly devs if I can narrow it down further, or feel free to do that yourself in a bug report to them.

Closing this ticket as I don't think there's anything more we can do on our side.

Note: See TracTickets for help on using tickets.