Opened 6 years ago

Closed 6 years ago

#1636 closed defect (fixed)

i2psnark fills up tmp

Reported by: DISABLED Owned by: zzz
Priority: major Milestone: 0.9.22
Component: apps/i2psnark Version: 0.9.21
Keywords: i2psnark tmp files Cc:
Parent Tickets: Sensitive: no


Hi there,

since I have been running 100+ torrents in i2psnark for a long time now, I guess this new behaviour is a bug. i2psnark creates lots of files in my tmp (/tmp/i2p-*.tmp/i2psnark-*/piece_*.tmp). My tmp (tmpfs 50% RAM) is limited to 6gb so that kills my server. Used to have no problem there.

I guess these files are created in PartialPiece?: tempfile.

In fact at least some files DO get deleted (just added some logging). Maybe there are just too many or some stay around - don't know yet.

I hope this is not a new feature which will require my tmp to be as large as my snark-folder! Because that is usually around 300gb.

I2P version: 0.9.21-0
Java version: Oracle Corporation 1.8.0_45-internal (OpenJDK Runtime Environment 1.8.0_45-internal-b14)
Wrapper version: 3.5.25
Server version: 8.1.17.v20150415
Servlet version: Jasper JSP 2.1 Engine
Platform: Linux amd64 3.19.0-25-generic
Processor: Haswell (22nm) (corei)
Jbigi: Locally optimized native BigInteger? library loaded from file
Encoding: UTF-8
Charset: UTF-8

No messages in the logs…


Change History (8)

comment:1 Changed 6 years ago by zzz

possibly related to #1633

comment:2 Changed 6 years ago by DISABLED

Add a subticket #1637.

comment:3 Changed 6 years ago by DISABLED

Add a subticket #1638.

comment:4 Changed 6 years ago by killyourtv

"guest" posted this in other tickets:

Found no other way for "guest" to add a comment to ticket 1636.

My workaround is to set in PartialPiece??:

private static final int MAX_IN_MEM = Integer.MAX_VALUE;
private static final int _max_in_mem = Integer.MAX_VALUE;

I prefer having the partial pieces in java heap than in tmpfs. That should be less overhead (and better performance). My router starts/stops torrents depending on load average, so performance does count for me.
Had to extend max-memory to 700M. It used to be >1000M some time ago. And I REALLY wondered how and why i2psnark was suddenly using less memory… now I know! :)
I understand that moving java heap to tmp files makes it easier for the inexperienced user to run i2psnark. But for me using heap seems a better solution.

I found two suspects which might leave the tmp-files there:
There is a outstandingRequests.remove(0); in PeerState??.getOutstandingRequest()
I don't understand the logic behind removing the first entry anyways. Commenting that out made no harm, but also the leak was still there. But that is still a hot candidate for me.

And then there is a outstandingRequests.clear(); in PeerState??.List() which also does not call the release() on PartialPiece??.

There are some other really strange things done Integers, indexes, etc. I'd say that will be a tough leak to find :) Comparing version 0.9.20 didn't supply any hint for me :(

and in another

PS: Might be a coincidence, but after removing outstandingRequests.remove(0); my download rates got much higher than usual, like 300-400kb instead of 100-200kb.

comment:5 Changed 6 years ago by killyourtv

guest should create his/her own account since "guest" should be inactive due to spam. (It's disabled now).

comment:6 Changed 6 years ago by zzz

6 GB of temp files sounds incredible but let's do the math…

100 torrents (assuming all leaching) each with several peers, say working on 8 pieces at once, assuming all large 1 MB pieces, half complete, thats 400 MB. Long way from 6 GB.

If there is some temp file leak then perhaps the files would stick around after snark was stopped… or maybe it would clean them up.

As I said above, possibly related to #1633. Will attempt to reproduce here. A quick glance in my temp dir didn't show any leftover files at the moment.

comment:7 Changed 6 years ago by zzz

Milestone: undecided0.9.22
Status: newtesting

#1633 fix is in 0.9.21-2. At http://zzz.i2p/topics/1930 a user reports thatit appears to have fixed or reduced the /tmp problem in this ticket.

Optimistically moving this ticket to 'testing' also.

comment:8 Changed 6 years ago by zzz

Resolution: fixed
Status: testingclosed
Note: See TracTickets for help on using tickets.