Opened 6 years ago

Last modified 18 months ago

#1146 infoneeded enhancement

High "system" CPU usage by SimpleTimer2 threads

Reported by: Zlatin Balevsky Owned by: zzz
Priority: minor Milestone:
Component: router/general Version: 0.9.9
Keywords: windows win64 x86-64 mac performance Cc: zab@…, zzz@…, dg2@…
Parent Tickets: Sensitive: no

Description

Analysis shows that almost half of the CPU usage on Windows is due to SimpleTimer2 threads bumping in each other.

Performance with:
4 SimpleTimer2 threads: http://imgur.com/a/esadK#3
2 threads: http://imgur.com/a/ivXMX#2

Subtickets

Change History (7)

comment:1 Changed 6 years ago by Zlatin Balevsky

Windows 7 x64
Oracle 1.7 64 bit

comment:2 Changed 6 years ago by zzz

Is your theory that the SimpleTimer2 implementation (ScheduledThreadPoolExecutor? and its queues and ST2.TimedEvent?.run()) are the problem, or that the TimedEvents? that run on ST2 are "bumping" in their timedReached() ?

If the former, I don't have a lot of ideas. Is this limited to Windows?

If the latter, we should try to get to the root cause.

One possible example of the latter is FragmentHandler? where it queues a timer for every received msg at the (OB or IB) EP. A periodic cleaner could be more efficient. There's other cases I'm sure.

We don't have any visibility into ST2 usage. Something that measures run counts, total run time, and max run time, by TimedEvent? superclass (perhaps using StatisticsManager?, perhaps just a HashMap? local to ST2), could aid in the diagnosis.

So atm I view the 4→2 change as more of a pointer toward the problem than a solution.

comment:3 Changed 6 years ago by Zlatin Balevsky

SimpleTimer2 implementation. There is a single queue inside the ScheduledThreadPoolExecutor and the worker threads are either:
a) bumping in each other trying to dequeue the next TimedEvent
b) all waking up when there is only one event in the queue, and all but one are going to sleep again

Reducing the number of threads reduces the contention and idle thread activity in both of these situations. I'm observing similar CPU usage reduction in OSX with Java 1.6.

comment:4 Changed 5 years ago by str4d

Keywords: windows win64 x86-64 mac performance added; CPU Windows removed
Milestone: 0.9.10

comment:5 Changed 4 years ago by str4d

Status: newopen

comment:6 Changed 18 months ago by Eche|on

Owner: set to zzz
Status: openassigned

comment:7 Changed 18 months ago by zzz

Status: assignedinfoneeded

There's been several changes to ST2 since this ticket, some written by the OP.
There's also the changes for #1672 in 0.9.23
And now we have Java 8.
Could OP please retest?

Note: See TracTickets for help on using tickets.