Opened 7 years ago

Closed 6 years ago

#686 closed defect (fixed)

"GC loop of death" example with 0.9.1-5

Reported by: Zlatin Balevsky Owned by: HungryHobo
Priority: minor Milestone:
Component: apps/plugins Version: 0.9.1
Keywords: OOM Cc: sponge
Parent Tickets: Sensitive: no

Description

"user" on #i2p-dev has been kind enough to forfeit his anonymity and provide us with a heap dump after observing the "GC Loop Of Death". Heap dump for jhat analysis from his router as well as wrapper logs are available here:

http://t5qds7twb7eyyvp2fchhbn74tgdzv774rlru5guit5jkn4a6do7a.b32.i2p/dump/

description of the gc loop of death available here
http://pastethis.i2p/show/1587/

a preliminary look at the logs suggests seedless may be to blame, but with OOMs you never know.

Subtickets

Change History (8)

comment:1 Changed 7 years ago by killyourtv

I fetched the dump and FWIW the first 42 entries of the histogram are

Class Instance Count Total Size
class [B 415234 284059272
class [Ljava.lang.Object; 146838 20450016
class [C 199208 19903746
class [I 180329 10263944
class [Ljava.util.HashMap$Entry; 60797 8654712
class [Ljava.util.concurrent.ConcurrentHashMap$HashEntry; 259171 8610488
class java.util.concurrent.ConcurrentHashMap$Segment 259168 8293376
class java.util.concurrent.locks.ReentrantLock$NonfairSync 266021 7448588
class net.i2p.client.streaming.Connection 19047 6437886
class net.i2p.stat.Rate 53111 6373320
class java.util.concurrent.ConcurrentHashMap$HashEntry 155459 4352852
class java.lang.String 215583 4311660
class [S 43647 3416862
class java.util.HashMap$Entry 112094 3138632
class net.i2p.client.streaming.ConnectionOptions 19079 3071719
class java.util.HashMap 57063 2739024
class java.util.concurrent.FutureTask$Sync 39978 2718504
class [Ljava.util.concurrent.ConcurrentHashMap$Segment; 16198 2332512
class net.i2p.client.streaming.MessageOutputStream 19047 1999935
class java.util.Hashtable$Entry 62701 1755628
class org.neodatis.odb.core.layers.layer3.BytesImpl2 72351 1736424
class org.neodatis.odb.core.layers.layer2.meta.AttributeIdentification 79668 1673028
class org.neodatis.odb.core.layers.layer2.meta.NullNativeObjectInfo 57469 1609132
class net.i2p.client.streaming.MessageInputStream 19047 1561854
class net.i2p.data.ByteArray 92438 1479008
class [Ljava.util.Hashtable$Entry; 9373 1301800
class [[I 9523 1291168
class java.util.TreeMap 19145 1225280
class java.io.IOException 37474 1199168
class java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask 21149 1015152
class net.i2p.data.Destination 29987 959584
class [J 925 941560
class java.util.concurrent.ConcurrentHashMap 16164 905184
class net.i2p.data.Hash 36962 887088
class java.lang.Long 109666 877328
class net.i2p.router.transport.udp.PeerState 2097 859770
class java.util.ArrayList 48800 780800
class net.i2p.client.streaming.MessageOutputStream$Flusher 19047 704739
class net.i2p.util.Log 15688 690272
class net.i2p.client.streaming.Connection$ActivityTimer 19047 685692
class [Lorg.neodatis.odb.core.layers.layer2.meta.AbstractObjectInfo; 2350 674944
class [Lorg.neodatis.odb.core.layers.layer2.meta.AttributeIdentification; 2350 674944

comment:2 Changed 7 years ago by Zlatin Balevsky

The first very wrong thing that the heap analysis reveals is:

18,820 strongly reachable instances of i2p.bote.network.I2PPacketDispatcher$StreamReceiveTask?
shallow size 451,680 bytes
retained size 171MB

Looks like another case of unbounded queue, except inside a threadpool executor this time.

comment:3 Changed 7 years ago by Zlatin Balevsky

Another thing that looks wrong, could be related to bote:

19,047 reachable instances of net.i2p.client.streaming.MessageInputStream?
shallow size 1.3MB
retained size 153MB

the _readyDataBlocks list inside each stream seems to be unbounded as well.

comment:4 Changed 7 years ago by Zlatin Balevsky

This will most likely get resolved with #690

comment:5 Changed 7 years ago by zzz

Component: unspecifiedapps/plugins
Owner: set to HungryHobo
Status: newassigned

whether or not the fix is in I2PTunnel (as you're probably implying with #690), HH should look at this also, to see what may need to be done on his side.

comment:6 Changed 7 years ago by zzz

Cc: sponge added

cc'ing sponge due to mention of seedless above

comment:7 Changed 7 years ago by zzz

… and as with #690, link above is dead

comment:8 Changed 6 years ago by HungryHobo

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.