Opened 8 months ago

Closed 6 months ago

#2234 closed enhancement (fixed)

Sequential Downloads in I2PSnark

Reported by: slumlord Owned by: zzz
Priority: minor Milestone: 0.9.36
Component: apps/i2psnark Version: 0.9.34
Keywords: i2psnark, sequential, stream Cc:
Parent Tickets:

Description (last modified by slumlord)

Searched for a ticket regarding this, I thought I had opened one. Perhaps I had only posted about it on zzz.i2p.

---

I think this would be a useful feature to have in I2PSnark - it would allow a user to essentially stream well-seeded torrents of video files, either on a per-file or per-torrent basis. It could be handled similarly to the file priority settings. One possiblity is to have a checkbox beside each file for "Sequential Download" - an unchecked box would have I2PSnark download the chunks corresponding to that file randomly, as normal, while a checked box would have I2PSnark download the chunks corresponding to that file sequentially. For video file formats such as .mkv and .mp4, the user will be able to start watching the video file as soon as sufficient chunks have been "buffered".

An example of some well-seeded torrents which theoretically can be downloaded fast enough to be streamed - Videos from the 33rd Chaos Communication Congress, transcoded to x265 to reduce filesize: link removed, attached as file instead.

Subtickets

Attachments (1)

33c3_videos.x265.txt (30.0 KB) - added by slumlord 8 months ago.
Recorded videos from 33C3, transcoded to x265

Download all attachments as: .zip

Change History (13)

comment:1 Changed 8 months ago by slumlord

  • Description modified (diff)

comment:2 Changed 8 months ago by slumlord

  • Description modified (diff)

Changed 8 months ago by slumlord

Recorded videos from 33C3, transcoded to x265

comment:3 Changed 8 months ago by zzz

yeah.

Separate but related, or maybe it's the same thing, is setting the file priorities to be in alphabetical order, so if you get a whole season of videos or a whole album of music, you get the first one completed first.

The UI for my idea would be just one button, not one per file, so it wouldn't be as much of a mess... maybe... but it wouldn't fit into the high/low/skip scheme. Would it replace it, or would one override the other? don't know.

For sequential, one setting for the whole torrent would be cleaner also.

possibly related: #44 #1026

comment:4 Changed 8 months ago by slumlord

It would probably be easier to set a "Sequential Download" flag for the whole torrent. As far as files and their download priorities are concerned, perhaps it would be best to download all chunks sequentially if the Sequential Download flag is set and for any files that are set to "do not download", those chunks are just ignored (allowing for chunks that are part of multiple files, of course).

comment:5 Changed 8 months ago by zzz

Right, so the simplest UI would be a whole-torrent flag that does both: Alphabetical file priority, and in-order piece priority within each file. The only per-file choice would be skip, perhaps as a checkbox instead of radio

comment:6 Changed 8 months ago by slumlord

That makes sense to me.


More well-seeded videos for testing
Recorded videos from I2PCon (x265):

I2PCon-2015-x265.mkv magnet:?xt=urn:btih:ea74c208e8ba7801be1314100f454332215f24fb 955M 
├── [   61136549]  I2PCon-2015-x265.mkv/20150815_01Intro_I2PCon.x265.mkv
├── [  140802255]  I2PCon-2015-x265.mkv/20150815_02NickJohnston_TOSvsPII_I2PCon.x265.mkv
├── [  155310020]  I2PCon-2015-x265.mkv/20150815_03zzz_GrowingNetworkSpreadingWord_I2PCon.x265.mkv
├── [  153553448]  I2PCon-2015-x265.mkv/20150815_04PanelDiscussion_I2PCon.x265.mkv
├── [  151619493]  I2PCon-2015-x265.mkv/20150816_05DavidDagon_I2PMonitoringFiltration_I2PCon.x265.mkv
├── [  137729327]  I2PCon-2015-x265.mkv/20150816_06ConnectionLimitChallenge_I2PCon.x265.mkv
├── [   72425041]  I2PCon-2015-x265.mkv/20150816_07LanceJames_I2PCon.x265.mkv
├── [   53552195]  I2PCon-2015-x265.mkv/20150816_08Psi_I2PAppDevIntro.x265.mkv
└── [   74787911]  I2PCon-2015-x265.mkv/20150816_09str4d_AgonyEcstasyofI2POnAndroid_I2PCon.x265.mkv

comment:7 Changed 7 months ago by slumlord

Additional thought: Allow the usage of a parameter in an I2P torrent magnet which, when specified, automatically has I2PSnark set the 'Sequential Download' flag.

references:
https://en.wikipedia.org/wiki/Magnet_URI_scheme
https://en.wikipedia.org/wiki/Magnet_URI_scheme#Supplement_format_(x.)

For example, if we use the xq parameter

magnet:?xt=urn:btih:19e3f830707194eb5d9803484a9c58aa7564df07&xq=1

Adding a magnet of this form, would set the Sequential Download flag when added to I2PSnark - assume 0 if flag is not set. There could also be a config option to set all torrents to download sequentially.

also: http://bittorrent.org/beps/bep_0009.html

comment:8 Changed 7 months ago by zzz

  • Milestone changed from undecided to 0.9.36
  • Status changed from new to accepted

re: comment 7, I don't know who would add a x.q parameter for us
Optimistically setting target for .36 for whole-torrent flag as in comment 5

comment:9 Changed 6 months ago by zzz

  • Resolution set to fixed
  • Status changed from accepted to closed

Implemented as described in comments 3-5 above, in dfe35cbf2e20ebea776e8ba55fa83fa3390b60d2 and previous revisions, 0.9.35-11

comment:10 Changed 6 months ago by zzz

  • Resolution fixed deleted
  • Status changed from closed to reopened

As slumlord discovered, option is hidden if only one file, thus preventing in-order-pieces for a single-file torrent.

Reopening for discussion on changing that (and would the option text change for the single-file case as well?)

Also, the in-order fetching is of course best effort only. If the option is enabled, should the file completion percentages show "usable" completion (distance to first unreceived piece) instead?

comment:11 Changed 6 months ago by slumlord

Reopening for discussion on changing that (and would the option text change for the single-file case as well?)

Makes sense to me to have this behavior for single-file torrents. The 33C3 videos I shared for testing are all single-file torrents. Option text could be changed to Download file(s) in order:.

Also, the in-order fetching is of course best effort only. If the option is enabled, should the file completion percentages show "usable" completion (distance to first unreceived piece) instead?

I think the current behavior is fine and can be kept as-is.

---

Edit: Forgot to mention that I tested this with a multi-file torrent and it works well.

Last edited 6 months ago by slumlord (previous) (diff)

comment:12 Changed 6 months ago by zzz

  • Resolution set to fixed
  • Status changed from reopened to closed

Enabled for single-file torrents in 585e8aee4b544a728e9a4354058b7408b7580311 0.9.35-15

Note: See TracTickets for help on using tickets.