#2471 closed defect (fixed)

OOM in net.i2p.util.ResettableGZIPOutputStream

Reported by: zzz Owned by:
Priority: minor Milestone: 0.9.40
Component: api/utils Version: 0.9.39
Keywords: Cc:
Parent Tickets: Sensitive: no

Description

From parg via http://zzz.i2p/topics/2700

I've got a user who's BiglyBT repeatedly crashes with OOM

[01:53:23] java.lang.OutOfMemoryError
at java.util.zip.Deflater.init(Native Method)
at java.util.zip.Deflater.<init>(Unknown Source)
at net.i2p.util.ResettableGZIPOutputStream.<init>(ResettableGZIPOutputStream.java:27)
at net.i2p.util.ReusableGZIPOutputStream.<init>(ReusableGZIPOutputStream.java:61)
at net.i2p.util.ReusableGZIPOutputStream.acquire(ReusableGZIPOutputStream.java:43)
at net.i2p.data.DataHelper.compress(DataHelper.java:1714)
at net.i2p.data.DataHelper.compress(DataHelper.java:1693)
at net.i2p.client.impl.I2PSessionMuxedImpl.prepPayload(I2PSessionMuxedImpl.java:295)
at net.i2p.client.impl.I2PSessionMuxedImpl.sendMessage(I2PSessionMuxedImpl.java:245)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.sendMessage(DHTTransportI2PRealImpl.java:2232)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.sendResponse(DHTTransportI2PRealImpl.java:2163)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveFindValue(DHTTransportI2PRealImpl.java:1310)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveQuery(DHTTransportI2PRealImpl.java:2500)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.receiveMessage(DHTTransportI2PRealImpl.java:2360)
at org.parg.azureus.plugins.networks.i2p.vuzedht.DHTTransportI2PRealImpl.messageAvailable(DHTTransportI2PRealImpl.java:2288)
at net.i2p.client.impl.I2PSessionDemultiplexer.messageAvailable(I2PSessionDemultiplexer.java:43)
at net.i2p.client.impl.I2PSessionMuxedImpl$MuxedAvailabilityNotifier.run(I2PSessionMuxedImpl.java:408)
at java.lang.Thread.run(Unknown Source)
at net.i2p.util.I2PThread.run(I2PThread.java:103)

From logs he appears to have lots (GBs) of JVM heap available around this time. I'm also seeing periodic high CPU usage in the finalization of Deflate objects:

[14:15:55] Thread state: elapsed=10001,cpu=16754,max=Finalizer(7281/72%),mem:max=7339584,tot=523840,free=355445
[14:15:55] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
[14:16:05] Thread state: elapsed=10001,cpu=11667,max=Finalizer(8813/88%),mem:max=7339584,tot=523840,free=288464
[14:16:05] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
[14:16:15] Thread state: elapsed=10001,cpu=12345,max=Finalizer(8718/87%),mem:max=7339584,tot=523840,free=346504
[14:16:15] java.util.zip.Deflater.end(Native Method), java.util.zip.Deflater.end(Unknown Source), java.util.zip.Deflater.finalize(Unknown Source), java.lang.System$2.invokeFinalize(Unknown Source), java.lang.ref.Finalizer.runFinalizer(Unknown Source), java.lang.ref.Finalizer.access$100(Unknown Source), java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

Is this possibly related to a similar issue I came across in another app - https://github.com/jwtk/jjwt/issues/392

User is running Java 1.8.0_152 (64 bit) and 0.9.39

Subtickets

Change History (3)

comment:1 Changed 18 months ago by zzz

We used to cache the ReusableGZIPOutputStream objects, but we were getting persistent corrupted compressed routerinfos over the network, and could never figure out the cause. See #1915. In desperation we disabled the caching two years ago for 0.9.29.

I still see occasional corrupted routerinfos. I used to have some debugging locally to see what version they were, can't recall the details but I believe they were all old versions.

These issues are really tough to diagnose because it's not related to the JVM version but to the JVM's underlying native-code zlib library which varies by OS and platform and version.

comment:2 Changed 18 months ago by zzz

I will look to see if we're closing the streams, that seems to be the takeaway from the link in the OP.

comment:3 Changed 18 months ago by zzz

Milestone: undecided0.9.40
Resolution: fixed
Status: newclosed

Fixed as suggested by parg in zzz.i2p thread
in 8109f9041b31355f5e1f788378f43f20a4ef22b3 to be 0.9.39-5

Note: See TracTickets for help on using tickets.