Opened 6 years ago

Closed 4 years ago

#1099 closed defect (fixed)

java.io.EOFException in I2PSnark on large (16+ gb) torrents

Reported by: xmz Owned by: zzz
Priority: minor Milestone: 0.9.9
Component: apps/i2psnark Version: 0.9.8.1
Keywords: heisenbug Cc:
Parent Tickets:

Description

I2P Version and Running Environment:

I2P version: 0.9.8.1-0
Java version: Oracle Corporation 1.7.0_25 (OpenJDK Runtime Environment 1.7.0_25-b15)
Wrapper version: 3.5.19
Server version: 7.6.12.v20130726
Servlet version: Jasper JSP 2.1 Engine
Platform: FreeBSD amd64 9.1-RELEASE
Processor: uninitialized (pentiummmx)
Jbigi: Locally optimized native BigInteger library loaded from file
Encoding: US-ASCII
Charset: US-ASCII

When seeding a large torrent file on I2PSnark (two actually, one of 16.66 GB and the other of 17.44 GB) I encounter the error

Error reading the storage (piece 192) for PART_1: java.io.EOFException
Fatal storage error: Stopping torrent PART_2
Error reading the storage (piece 2054) for PART_2: java.io.EOFException
Starting up torrent PART_2

in I2PSnark.

To rule out filesystem errors I have run a checksum test on the files contained in the two torrents, no errors were found. The files pass the integrity test.

Router logs are pasted here: Router Logs

The wrapper logs do not seem to have anything regarding this issue but I can share them if required.

Subtickets

Change History (18)

comment:1 Changed 6 years ago by zzz

  • Status changed from new to infoneeded_new

Need a link to the torrent files so I can try it here. If you'd rather keep it private you may PM it to me on IRC or email it to me zzz@….

If it's always the same file (I assume there are multiple files in the torrents) please indicate which file.

comment:2 Changed 6 years ago by xmz

  • Status changed from infoneeded_new to new

Here are the torrents:

http://tracker2.postman.i2p/index.php?view=TorrentDetail&id=23803
http://tracker2.postman.i2p/index.php?view=TorrentDetail&id=23805

I can't tell which file causes the issue. I2PSnark responds with a piece number. I don't know how to correlate the piece number to file. If there is a way, let me know.

Last edited 6 years ago by xmz (previous) (diff)

comment:3 Changed 6 years ago by zzz

Lets look at the first one since it's a low piece:

  java.lang.RuntimeException: Error reading the storage (piece 192) for PART_1

It's 4MB pieces so that's about 800 MB into the torrent. That's less than the size of the first file:

   linuxmint-15-cinnamon-dvd-64bit.iso_dir/linuxmint-15-cinnamon-dvd-64bit.iso (959447040)

and that makes any math problems less likely.

Also, this isn't the first torrent over 2GB (overflowing an int) or with a file larger than 2GB.

The storage code was substantially rewritten for 0.9.9 so I tried both 0.9.8.1 release and the latest in trunk. I added the torrents, then stopped I2P, then touched all the files to force a reread on startup, then started I2P. All was fine on both 0.9.8.1 and trunk.

I did see a minor problem where we don't check the file length on startup unless the file modification date changes. I'll fix that in trunk.

I know you said you did a checksum test. But as a double check please verify the length and permissions of the file listed above.

comment:4 Changed 6 years ago by xmz

I think I know what is causing this. The files in this torrent are compiled from a number of folders that reside at ~/downloads/.
I created symlinks to the folders in ~/downloads at ~/.i2p/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_{1,2}/. Then I created the large torrent. The thing that must be causing this issue is that I am seeding each of the folders in PART_{1,2} separately, as their own torrent. For example, PART_1 has a folder for linuxmint-15-cinnamon-dvd-64bit.iso_dir. The files in the folder are referenced by two different torrents which eventually map to the same location on the disk.
I2PSnark has a symlink to the files in ~/downloads/linuxmint-15-cinnamon-dvd-64bit.iso_dir/ from both ~/.i2p/i2psnark/linuxmint-15-cinnamon-dvd-64bit.iso_dir/ and ~/.i2p/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_1/linuxmint-15-cinnamon-dvd-64bit.iso_dir/

I did not think this would be a problem but I guess the file must be locked for reading from the torrent that was created first and a seperate read attempt from the pack fails.

Additionally, I have checked the file size and permissions and do not see anything wrong.

What is the desired behaviour in this case? Should the user be able to seed two different torrents off of the same file?

Last edited 6 years ago by xmz (previous) (diff)

comment:5 Changed 6 years ago by zzz

You have the same file in two different torrents, or just symlinks to two different subdirectories? I only see linuxmint in part 1, for example.

Even so, if they are both seeds, it should be ok. Unless I am missing something.

Separately, I'll be fixing snark so the name of the file that causes an IOException will be put in the logs, not just the piece number.

comment:6 Changed 6 years ago by xmz

Right, let me explain.

~/.i2p/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_1/linuxmint-15-cinnamon-dvd-64bit.iso_dir/ is just a symlink to ~/downloads/linuxmint-15-cinnamon-dvd-64bit.iso_dir/ where the files reside. So in the torrent data folder for PART_1, I have symlink to the actual files which are located in ~/downloads/linuxmint-15-cinnamon-dvd-64bit.iso_dir/.

At the same time I also have a torrent - linuxmint-15-cinnamon-dvd-64bit.iso_dir - in i2psnark, which is a symlink at ~/.i2p/i2psnark/linuxmint-15-cinnamon-dvd-64bit.iso_dir/ which also points to different ~/downloads/linuxmint-15-cinnamon-dvd-64bit.iso_dir/. That torrent can be found here.

So, the data is referenced by two torrents through symbolic links in two locations. Data is in one place only: ~/downloads/ but I have symlinks in two places - ~/.i2p/i2psnark/PART_{1,2} and ~/.i2p/i2psnark/ which correspond to torrents. Each folder contained in PART_1 and PART_2 has a seperate symlink pointing to data stored in ~/downloads

I have the same file in two different torrents through symlinks.

I am only seeding all of these torrents there is no downloading, the data has not been touched since the first torrents were created.

Hope this makes sense.

comment:7 Changed 6 years ago by zzz

OK, so give it a try without the symlinks and see if it fixes it.

comment:8 Changed 6 years ago by xmz

I will try this.

I would like to say that, if this clears the issue, it would not be an optimal solution as I would have to use twice the amount of space.

I will update after testing.

comment:9 Changed 6 years ago by xmz

Replaced symlinks with actual files but I still get:

Error reading the storage (piece 1181) for PART_1: java.io.EOFException

In I2PSnark, all files show as being 0 B in size. I had not noticed this before.

I have recreated the torrent after replacing the symlinks with the actual files. I then tried to seed the files to see if the same problem would appear and also to see what size I2PSnark reports for the files. I had to remove the first torrent I created for PART_1 in order to make an updated one from the non-symlinked files. In the updated torrent, file sizes still show as being 0 B, proof(clearnet). Again, this is from the actual files. There are no symlinks anymore. The torrent is at: Postman. The torrent is seeding, I will update if the error occurs again.


Last edited 6 years ago by xmz (previous) (diff)

comment:10 Changed 6 years ago by zzz

Sounds like a permissions problem. Or, a '/' in the name of the torrent. The slash is likely a problem.

comment:11 Changed 6 years ago by xmz

Error reading the storage (piece 1399) for PART_1: java.io.EOFException

After a few hours of running the torrent, the same issue is encountered.

11/4/13 4:56:47 PM ERROR [@pivylM 2581] .klomp.snark.PeerConnectionOut: Error sending to AwMD@pivylM 2581
     java.lang.RuntimeException: Error reading the storage (piece 1399) for PART_1: java.io.EOFException
     at org.klomp.snark.PeerCoordinator.gotRequest(PeerCoordinator.java:926)
     at org.klomp.snark.PeerState.loadData(PeerState.java:250)
     at org.klomp.snark.Message.sendMessage(Message.java:93)
     at org.klomp.snark.PeerConnectionOut.run(PeerConnectionOut.java:180)
     at java.lang.Thread.run(Thread.java:724)
     at net.i2p.util.I2PThread.run(I2PThread.java:85)
     Caused by: java.io.EOFException
     at java.io.RandomAccessFile.readFully(RandomAccessFile.java:416)
     at org.klomp.snark.Storage.getUncheckedPiece(Storage.java:1150)
     at org.klomp.snark.Storage.getPiece(Storage.java:989)
     at org.klomp.snark.PeerCoordinator.gotRequest(PeerCoordinator.java:915)
     ... 5 more
11/4/13 4:56:47 PM ERROR [@pivylM 2581] rg.klomp.snark.PeerCoordinator: Error reading the storage (piece 1399) for PART_1: java.io.EOFException
     java.io.EOFException
     at java.io.RandomAccessFile.readFully(RandomAccessFile.java:416)
     at org.klomp.snark.Storage.getUncheckedPiece(Storage.java:1150)
     at org.klomp.snark.Storage.getPiece(Storage.java:989)
     at org.klomp.snark.PeerCoordinator.gotRequest(PeerCoordinator.java:915)
     at org.klomp.snark.PeerState.loadData(PeerState.java:250)
     at org.klomp.snark.Message.sendMessage(Message.java:93)
     at org.klomp.snark.PeerConnectionOut.run(PeerConnectionOut.java:180)
     at java.lang.Thread.run(Thread.java:724)
     at net.i2p.util.I2PThread.run(I2PThread.java:85)

Torrent is created through web front-end by passing the parameter TOP_25_FREE_OPERATING_SYSTEMS/PART_1/ to I2PSnark and clicking "create torrent" button.

Yes, I accidentally deleted the archlinux-2013.09.01-dual.iso_dir directory and I had to download it. The permissions were wrong, as you said. I have fixed the permissions so it can be read by any user.

UPDATE-
It failed again at the same point:

Fatal storage error: Stopping torrent PART_1
Error reading the storage (piece 1399) for PART_1: java.io.EOFException
Last edited 6 years ago by xmz (previous) (diff)

comment:12 Changed 6 years ago by xmz

Since there was a problem with file permissions, should I try to recreate the torrent? The java.io.EOFException is persisting otherwise.

comment:13 Changed 6 years ago by xmz

I fixed all permissions and created the torrent again. It fails at piece 1399 again.
Permissions on files:

.
├── [drwxr--r--]  CentOS-6.4-x86_64-LiveDVD
│   ├── [-rw-r--r--]  CentOS-6.4-x86_64-LiveDVD.iso
│   ├── [-rw-r--r--]  CentOS-6.4-x86_64-LiveDVD.sfv
│   ├── [-rw-r--r--]  centos.org
│   ├── [-rw-r--r--]  md5sum.txt
│   ├── [-rw-r--r--]  md5sum.txt.asc
│   ├── [-rw-r--r--]  sha1sum.txt
│   ├── [-rw-r--r--]  sha1sum.txt.asc
│   ├── [-rw-r--r--]  sha256sum.txt
│   └── [-rw-r--r--]  sha256sum.txt.asc
├── [drwxr--r--]  Fedora-Live-Desktop-x86_64-19-1.iso_dir
│   ├── [-rw-r--r--]  Fedora-Live-Desktop-x86_64-19-1.iso
│   ├── [-rw-r--r--]  Fedora-Live-Desktop-x86_64-19-1.iso_dir.sfv
│   └── [-rw-r--r--]  fedoraproject.org
├── [drwxr--r--]  Mageia-3-x86_64-DVD
│   ├── [-rw-r--r--]  DATE.txt
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.idx
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.iso
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.iso.md5
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.iso.md5.gpg
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.iso.sha1
│   ├── [-rw-r--r--]  Mageia-3-x86_64-DVD.iso.sha1.gpg
│   └── [-rw-r--r--]  Mageia-3-x86_64-DVD.sfv
├── [drwxr--r--]  archlinux-2013.09.01-dual.iso_dir
│   ├── [-rw-r--r--]  archlinux-2013.09.01-dual.iso
│   ├── [-rw-r--r--]  archlinux-2013.09.01-dual.iso.md5
│   ├── [-rw-r--r--]  archlinux-2013.09.01-dual.iso_dir.sfv
│   └── [-rw-r--r--]  archlinux.org
├── [drwxr--r--]  debian-7.1.0-amd64-DVD-1.iso_dir
│   ├── [-rw-r--r--]  debian-7.1.0-amd64-DVD-1.iso
│   ├── [-rw-r--r--]  debian-7.1.0-amd64-DVD-1.iso.md5
│   └── [-rw-r--r--]  debian.org
├── [drwxr--r--]  linuxmint-15-cinnamon-dvd-64bit.iso_dir
│   ├── [-rw-r--r--]  linuxmint-15-cinnamon-dvd-64bit.iso
│   ├── [-rw-r--r--]  linuxmint-15-cinnamon-dvd-64bit.iso.md5
│   ├── [-rw-r--r--]  linuxmint-15-cinnamon-dvd-64bit.iso.sfv
│   └── [-rw-r--r--]  linuxmint.com
├── [drwxr--r--]  manjaro-xfce-0.8.7.1-x86_x64_dir
│   ├── [-rw-r--r--]  manjaro-0.8.7.1-md5sum.txt
│   ├── [-rw-r--r--]  manjaro-xfce-0.8.7.1-x86_64-pkgs.txt
│   ├── [-rw-r--r--]  manjaro-xfce-0.8.7.1-x86_64.iso
│   └── [-rw-r--r--]  manjaro-xfce-0.8.7.1-x86_x64.sfv
├── [drwxr--r--]  openSUSE-12.3-GNOME-Live-x86_64.iso_dir
│   ├── [-rw-r--r--]  openSUSE-12.3-GNOME-Live-x86_64.iso
│   ├── [-rw-r--r--]  openSUSE-12.3-GNOME-Live-x86_64.iso.md5
│   ├── [-rw-r--r--]  openSUSE-12.3-GNOME-Live-x86_64.iso.sfv
│   └── [-rw-r--r--]  opensuse.org
├── [drwxr--r--]  pclinuxos64-kde-minime-2013.07
│   ├── [-rw-r--r--]  pclinuxos.com
│   ├── [-rw-r--r--]  pclinuxos64-kde-minime-2013.07.iso
│   ├── [-rw-r--r--]  pclinuxos64-kde-minime-2013.07.md5sum
│   └── [-rw-r--r--]  pclinuxos64-kde-minime-2013.07.sfv
├── [drwxr--r--]  precise-5.7.1.iso_dir
│   ├── [-rw-r--r--]  md5sums.txt
│   ├── [-rw-r--r--]  precise-5.7.1.iso
│   ├── [-rw-r--r--]  precise-5.7.1.iso_dir.sfv
│   └── [-rw-r--r--]  puppylinux.com
├── [drwxr--r--]  ubuntu-13.10-desktop-amd64.iso_dir
│   ├── [-rw-r--r--]  ubuntu-13.10-desktop-amd64.iso
│   ├── [-rw-r--r--]  ubuntu-13.10-desktop-amd64.iso.md5
│   ├── [-rw-r--r--]  ubuntu-13.10-desktop-amd64.iso_dir.sfv
│   └── [-rw-r--r--]  ubuntu.com
└── [drwxr--r--]  zorin-os-7-core-64.iso_dir
    ├── [-rw-r--r--]  zorin-os-7-core-64.iso
    ├── [-rw-r--r--]  zorin-os-7-core-64.iso.md5
    ├── [-rw-r--r--]  zorin-os-7-core-64.iso_dir.sfv
    └── [-rw-r--r--]  zorin-os.com

12 directories, 55 files
Last edited 6 years ago by xmz (previous) (diff)

comment:14 Changed 6 years ago by zzz

shouldn't your directories be 755 and not 744?

If it isn't a permissions problem I have no idea. I'm stumped.

Assuming the part 1 torrent is still the one with infohash 42f99f4d3c22a7446d1ef26ada144d940ca74cfe I looked at piece 1399. It's near the end of this file:

pc 1399 off 0 len 4194304 st 910376606 len 4194304 PART_1/ubuntu-13.10-desktop-amd64.iso_dir/ubuntu-13.10-desktop-amd64.iso

It isn't at the end or crossing a file boundary or anything else that might trigger some sort of latent bug.

Out of ideas.

comment:15 Changed 6 years ago by xmz

Right, I changed the permissions to 755. I had thought 744 would be fine as I2P was running as the same user that owns the files, so it would be able to read the files.

I remade the torrent with folders set to 755. It is seeding now, I will update this comment with details if error crops up again. UPDATE: Error still produced.

Fatal storage error: Stopping torrent PART_1
Error reading the storage (piece 1399) for PART_1: java.io.EOFException

One thing I noticed, the files are located in ~/.i2p/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_1/ and when I create the torrent in I2PSnark giving it that directory, I find that I2PSnark makes another directory in ~/.i2p/i2psnark/ called PART_1 which has the same directory structure as ~/.i2p/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_1/ but all the files are zero bytes. What's going on here?

~/.i2p/i2psnark/PART_1/
├── [drwx------         512]  CentOS-6.4-x86_64-LiveDVD
│   ├── [-rw-------           0]  CentOS-6.4-x86_64-LiveDVD.iso
│   ├── [-rw-------           0]  CentOS-6.4-x86_64-LiveDVD.sfv
│   ├── [-rw-------           0]  centos.org
│   ├── [-rw-------           0]  md5sum.txt
│   ├── [-rw-------           0]  md5sum.txt.asc
│   ├── [-rw-------           0]  sha1sum.txt
│   ├── [-rw-------           0]  sha1sum.txt.asc
│   ├── [-rw-------           0]  sha256sum.txt
│   └── [-rw-------           0]  sha256sum.txt.asc
├── [drwx------         512]  Fedora-Live-Desktop-x86_64-19-1.iso_dir
│   ├── [-rw-------           0]  Fedora-Live-Desktop-x86_64-19-1.iso
│   ├── [-rw-------           0]  Fedora-Live-Desktop-x86_64-19-1.iso_dir.sfv
│   └── [-rw-------           0]  fedoraproject.org
├── [drwx------         512]  Mageia-3-x86_64-DVD
│   ├── [-rw-------           0]  DATE.txt
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.idx
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.iso
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.iso.md5
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.iso.md5.gpg
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.iso.sha1
│   ├── [-rw-------           0]  Mageia-3-x86_64-DVD.iso.sha1.gpg
│   └── [-rw-------           0]  Mageia-3-x86_64-DVD.sfv
├── [drwx------         512]  archlinux-2013.09.01-dual.iso_dir
│   ├── [-rw-------           0]  archlinux-2013.09.01-dual.iso
│   ├── [-rw-------           0]  archlinux-2013.09.01-dual.iso.md5
│   ├── [-rw-------           0]  archlinux-2013.09.01-dual.iso_dir.sfv
│   └── [-rw-------           0]  archlinux.org
├── [drwx------         512]  debian-7.1.0-amd64-DVD-1.iso_dir
│   ├── [-rw-------           0]  debian-7.1.0-amd64-DVD-1.iso
│   ├── [-rw-------           0]  debian-7.1.0-amd64-DVD-1.iso.md5
│   └── [-rw-------           0]  debian.org
├── [drwx------         512]  linuxmint-15-cinnamon-dvd-64bit.iso_dir
│   ├── [-rw-------           0]  linuxmint-15-cinnamon-dvd-64bit.iso
│   ├── [-rw-------           0]  linuxmint-15-cinnamon-dvd-64bit.iso.md5
│   ├── [-rw-------           0]  linuxmint-15-cinnamon-dvd-64bit.iso.sfv
│   └── [-rw-------           0]  linuxmint.com
├── [drwx------         512]  manjaro-xfce-0.8.7.1-x86_x64_dir
│   ├── [-rw-------           0]  manjaro-0.8.7.1-md5sum.txt
│   ├── [-rw-------           0]  manjaro-xfce-0.8.7.1-x86_64-pkgs.txt
│   ├── [-rw-------           0]  manjaro-xfce-0.8.7.1-x86_64.iso
│   └── [-rw-------           0]  manjaro-xfce-0.8.7.1-x86_x64.sfv
├── [drwx------         512]  openSUSE-12.3-GNOME-Live-x86_64.iso_dir
│   ├── [-rw-------           0]  openSUSE-12.3-GNOME-Live-x86_64.iso
│   ├── [-rw-------           0]  openSUSE-12.3-GNOME-Live-x86_64.iso.md5
│   ├── [-rw-------           0]  openSUSE-12.3-GNOME-Live-x86_64.iso.sfv
│   └── [-rw-------           0]  opensuse.org
├── [drwx------         512]  pclinuxos64-kde-minime-2013.07
│   ├── [-rw-------           0]  pclinuxos.com
│   ├── [-rw-------           0]  pclinuxos64-kde-minime-2013.07.iso
│   ├── [-rw-------           0]  pclinuxos64-kde-minime-2013.07.md5sum
│   └── [-rw-------           0]  pclinuxos64-kde-minime-2013.07.sfv
├── [drwx------         512]  precise-5.7.1.iso_dir
│   ├── [-rw-------           0]  md5sums.txt
│   ├── [-rw-------           0]  precise-5.7.1.iso
│   ├── [-rw-------           0]  precise-5.7.1.iso_dir.sfv
│   └── [-rw-------           0]  puppylinux.com
├── [drwx------         512]  ubuntu-13.10-desktop-amd64.iso_dir
│   ├── [-rw-------           0]  ubuntu-13.10-desktop-amd64.iso
│   ├── [-rw-------           0]  ubuntu-13.10-desktop-amd64.iso.md5
│   ├── [-rw-------           0]  ubuntu-13.10-desktop-amd64.iso_dir.sfv
│   └── [-rw-------           0]  ubuntu.com
└── [drwx------         512]  zorin-os-7-core-64.iso_dir
    ├── [-rw-------           0]  zorin-os-7-core-64.iso
    ├── [-rw-------           0]  zorin-os-7-core-64.iso.md5
    ├── [-rw-------           0]  zorin-os-7-core-64.iso_dir.sfv
    └── [-rw-------           0]  zorin-os.com

12 directories, 55 files
Last edited 6 years ago by xmz (previous) (diff)

comment:16 Changed 5 years ago by xmz

I tried creating these torrents again on a different machine, again making two parts as I read somewhere that the maximum torrent size is preferably 20 GB. I placed the symlinks to the data under ~/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS-PART_1/ & ~/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS-PART_2/ and made the torrents in I2PSnark.

This time I have not experienced the error messages mentioned in this ticket.

Previously, the two folders were ~/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_1/ & ~/i2psnark/TOP_25_FREE_OPERATING_SYSTEMS/PART_2/.
Maybe I2PSnark has a bug when we try to give it subfolders for torrent data?

comment:17 Changed 4 years ago by str4d

  • Keywords heisenbug added; i2snark big torrent removed
  • Milestone 0.9.9 deleted

comment:18 Changed 4 years ago by zzz

  • Milestone set to 0.9.9
  • Resolution set to fixed
  • Status changed from new to closed

I can't make much sense of the above after a year. There's just too much going on with permissions and symlinks. There was a minor fix referenced in comment 3 above. Declaring this fixed. Please open a new ticket if you can get a bug boiled down to a simple test case.

Note: See TracTickets for help on using tickets.