Opened 7 months ago

Closed 7 months ago

#2471 closed defect (fixed)

OOM in net.i2p.util.ResettableGZIPOutputStream

Reported by: zzz Owned by:
Priority: minor Milestone: 0.9.40
Component: api/utils Version: 0.9.39
Keywords: Cc:
Parent Tickets: Sensitive: no

Description

From parg via http://zzz.i2p/topics/2700

I've got a user who's BiglyBT repeatedly crashes with OOM

[01:53:23] java.lang.OutOfMemoryError
at java.util.zip.Deflater.init(Native Method)
at java.util.zip.Deflater.<init>(Unknown Source)
at net.i2p.util.ResettableGZIPOutputStream.<init>(ResettableGZIPOutputStream.java:27)
at net.i2p.util.ReusableGZIPOutputStream.<init>(ReusableGZIPOutputStream.java:61)
at net.i2p.util.ReusableGZIPOutputStream.acquire(ReusableGZIPOutputStream.java:43)
at net.i2p.data.DataHelper.compress(DataHelper.java:1714)
at net.i2p.data.DataHelper.compress(DataHelper.java:1693)
at net.i2p.client.impl.I2PSessionMuxedImpl.prepPayload(I2PSessionMuxedImpl.java:295)
at net.i2p.client.impl.I2PSessionMuxedImpl.sendMessage(I2PSessionMuxedImpl.java:245)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.sendMessage(DHTTransportI2PRealImpl.java:2232)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.sendResponse(DHTTransportI2PRealImpl.java:2163)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveFindValue(DHTTransportI2PRealImpl.java:1310)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveQuery(DHTTransportI2PRealImpl.java:2500)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveMessage(DHTTransportI2PRealImpl.java:2360)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.messageAvailable(DHTTransportI2PRealImpl.java:2288)
at net.i2p.client.impl.I2PSessionDemultiplexer.messageAvailable(I2PSessionDemultiplexer.java:43)
at net.i2p.client.impl.I2PSessionMuxedImpl$MuxedAvailabilityNotifier.run(I2PSessionMuxedImpl.java:408)
at java.lang.Thread.run(Unknown Source)
at net.i2p.util.I2PThread.run(I2PThread.java:103)

From logs he appears to have lots (GBs) of JVM heap available around this time. I'm also seeing periodic high CPU usage in the finalization of Deflate objects:

[14:15:55] Thread state: elapsed=10001,cpu=16754,max=Finalizer(7281/72%),mem:max=7339584,tot=523840,free=355445
[14:15:55] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
[14:16:05] Thread state: elapsed=10001,cpu=11667,max=Finalizer(8813/88%),mem:max=7339584,tot=523840,free=288464
[14:16:05] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
[14:16:15] Thread state: elapsed=10001,cpu=12345,max=Finalizer(8718/87%),mem:max=7339584,tot=523840,free=346504
[14:16:15] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

Is this possibly related to a similar issue I came across in another app - https://github.com/jwtk/jjwt/issues/392

User is running Java 1.8.0_152 (64 bit) and 0.9.39

Subtickets

Change History (3)

comment:1 Changed 7 months ago by zzz

We used to cache the ReusableGZIPOutputStream objects, but we were getting persistent corrupted compressed routerinfos over the network, and could never figure out the cause. See #1915. In desperation we disabled the caching two years ago for 0.9.29.

I still see occasional corrupted routerinfos. I used to have some debugging locally to see what version they were, can't recall the details but I believe they were all old versions.

These issues are really tough to diagnose because it's not related to the JVM version but to the JVM's underlying native-code zlib library which varies by OS and platform and version.

comment:2 Changed 7 months ago by zzz

I will look to see if we're closing the streams, that seems to be the takeaway from the link in the OP.

comment:3 Changed 7 months ago by zzz

Milestone: undecided0.9.40
Resolution: fixed
Status: newclosed

Fixed as suggested by parg in zzz.i2p thread
in 8109f9041b31355f5e1f788378f43f20a4ef22b3 to be 0.9.39-5

Note: See TracTickets for help on using tickets.