macOS i2P install doesn't use shipped jbigi

[Aniba]

Follow up of ​http://trac.i2p2.i2p/ticket/1865#comment:5

My i2P install on macOS does not use the native jbigi or jcpuid.

I2P v0.9.28
macOS 10.12.2

Log files attached.

The first i2P launch shows neither jbigi or jcpuid being loaded. The second launch shows jbigi being loaded from the file libjbigi.jnilib compiled as described in ticket #1865.

wrapper.log:

2016/12/23 10:37:34 | --> Wrapper Started as Daemon
2016/12/23 10:37:34 | Java Service Wrapper Community Edition 64-bit 3.5.25
2016/12/23 10:37:34 |   Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.
2016/12/23 10:37:34 |     http://wrapper.tanukisoftware.com
2016/12/23 10:37:34 | 
2016/12/23 10:37:34 | Launching a JVM...
2016/12/23 10:37:35 | Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
2016/12/23 10:37:35 | WrapperManager: Initializing...
2016/12/23 10:37:35 | Starting I2P 0.9.28-0
2016/12/23 10:37:35 | WARNING: Resource name [libjcpuid-x86-osx.jnilib] was not found
2016/12/23 10:37:35 | WARNING: Native CPUID library jcpuid not loaded - will not be able to read CPU information using CPUID
2016/12/23 10:37:35 | WARNING: Native BigInteger library jbigi not loaded - using pure Java - poor performance may result - see http://i2p-projekt.i2p/jbigi for help
2016/12/23 10:38:19 | TERM trapped.  Shutting down.
2016/12/23 10:38:40 | CRIT  [r 1 shutdown] net.i2p.router.Router         : Shutting down the router...
2016/12/23 10:38:40 | CRIT  [r 1 shutdown] net.i2p.router.Router         : Starting final shutdown(3)
2016/12/23 10:38:40 | CRIT  [r 1 shutdown] net.i2p.router.Router         : Shutdown(3) complete
2016/12/23 10:38:42 | <-- Wrapper Stopped
2016/12/23 10:40:33 | --> Wrapper Started as Daemon
2016/12/23 10:40:33 | Java Service Wrapper Community Edition 64-bit 3.5.25
2016/12/23 10:40:33 |   Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.
2016/12/23 10:40:33 |     http://wrapper.tanukisoftware.com
2016/12/23 10:40:33 | 
2016/12/23 10:40:33 | Launching a JVM...
2016/12/23 10:40:34 | Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
2016/12/23 10:40:34 | WrapperManager: Initializing...
2016/12/23 10:40:34 | Starting I2P 0.9.28-0
2016/12/23 10:40:34 | WARNING: Resource name [libjcpuid-x86-osx.jnilib] was not found
2016/12/23 10:40:34 | WARNING: Native CPUID library jcpuid not loaded - will not be able to read CPU information using CPUID
2016/12/23 10:40:34 | INFO: Locally optimized library libjbigi-osx-corei_64.jnilib loaded from file

[zzz]

Interesting. We have libjcpuid-x86_64-osx.jnilib but not libjcpuid-x86-osx.jnilib. Should be trying to load the _64 version.

[zzz]

Please provide full version info from the top of /logs in the console, and your Mac model.

[Aniba]

/logs console [Note that this instance is running a locally compiled native jbigi as described above]:

I2P version: 0.9.28-0
Java version: Oracle Corporation 1.8.0_112 (Java(TM) SE Runtime Environment 1.8.0_112-b16)
Wrapper version: 3.5.25
Server version: 8.1.21.v20160908
Servlet version: Jasper JSP 2.1 Engine
JSTL version: standard-taglib 1.2.0
Platform: Mac OS X x86_64 10.12.2
Jcpuid version: 0
Processor: uninitialized (unrecognized)
Jbigi: Locally optimized library libjbigi-osx-corei_64.jnilib loaded from file
Jbigi version: 4
GMP version: 6.1.1
Encoding: UTF-8
Charset: UTF-8

About This Mac: Mac mini (Mid 2010)

[Aniba]

Looking through i2p/lib/router.jar>net/i2p/router/tasks/InstallUpdate.class, I see the possibility that I messed up the update process by moving i2p/jbigi.jar to i2p/jbigi.jar.bak.

I do not have a current i2p/jbigi.jar file. I should probably test that first... ;)

ls -l i2p/jbigi.jar.bak 
-rw-r--r--  1 username  staff  9079302 Oct 20 11:29 i2p/jbigi.jar.bak

Would you suggest a fresh reinstall from i2pinstall_0.9.28.jar?

What's the most efficient way to retain my site-specific bandwidth, memory, and other settings if I wipe out my current installation and replace with a fresh install?

Also, I suggest modifying the update behavior to provide the latest jar files if they do not exist in the i2p/ directory.

[zzz]

you can just do a new install to a temp directory and then copy over what jars you want, if that's easier

please provide the output of:
java -jar /path/to/i2p.jar systemversion

[Aniba]

I created a fresh install from java -jar i2pinstall_0.9.28.jar and still see the same issues: no native jbigi or jcpuid.

It works when I copy over a hand-compiled-from-source libjbigi.jnilib to /Applications/i2p.

java -jar /Applications/i2p/lib/i2p.jar systemversion
64 bit   : true
Java 6   : true
Java 7   : true
Java 8   : true
Java 9   : false
Android  : false
Apache   : false
ARM      : false
Gentoo   : false
GNU      : false
Linux Svc: false
Mac      : true
OpenJDK  : false
Windows  : false
Wrapper  : false
x86      : true
Max mem  : 3817865216

java -jar /Applications/i2p/lib/i2p.jar cpuid
**Failed to retrieve CPUInfo. Please verify the existence of jcpuid dll/so**
JCPUID Version: 0
 **CPUInfo**
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.i2p.util.CommandLine.exec(CommandLine.java:68)
	at net.i2p.util.CommandLine.main(CommandLine.java:51)
Caused by: java.lang.UnsatisfiedLinkError: freenet.support.CPUInformation.CPUID.doCPUID(I)Lfreenet/support/CPUInformation/CPUID$CPUIDResult;
	at freenet.support.CPUInformation.CPUID.doCPUID(Native Method)
	at freenet.support.CPUInformation.CPUID.getCPUModelName(CPUID.java:270)
	at freenet.support.CPUInformation.CPUID.main(CPUID.java:329)
	... 6 more

[zzz]

Thanks for that. A few conclusions:

• We are correctly detecting that it's a mac, is x86, and is 64 bit
• The log line "Resource name [libjcpuid-x86-osx.jnilib] was not found" had me confused, since it wasn't x86_64. But it turns out we don't log the x86_64 lookup failure, only the fallback 32-bit one. I'll fix that soon.
• We shouldn't try a fallback to 32-bit for mac anyway, we don't have a file for that anymore. The libjcpuid-x86_64-osx.jnilib file is a fat binary that contains both 64- and 32-bit binaries, which i didn't realize. I'll fix that soon.
• If the CPUID detection fails, jbigi wasn't trying to load "none" or "none_64", even if it was x86. I'll fix that soon.

What isn't clear is why your Mac Mini doesn't like the libjcpuid-x86_64-osx.jnilib file we ship. I think the info on the platform we used to build it on was lost.

We also don't know, because of the above issues, whether any of the libjbigi-osx-*.jnilib files from jbigi.jar will work. You could try unzipping jbigi.jar, then copying them one at a time to libjbigi.jnilib to test. Once I fix the above, then we should automatically extract the jbigi-osx-none_64.jnilib file from jbigi.jar, which won't be the fastest but it's something.

[zzz]

Found another bug, which may be the root cause:

When we extract the jcpuid lib from jbigi.jar on a mac, we were saving it as libjcpuid.so, not libjcpuid.jnilib, where it wouldn't be found. Renaming it made things work a lot better. Will be fixing that one too.

[zzz]

I have all the above fixed and will check it in soon as 0.9.29-2. But I'm not sure it's going to help, if your Mac Mini isn't compatible with our shipped jcpuid binary.

Please do:

cd /Applications/i2p

Do you have a libjcpuid.so file? If so, please do the following and provide the output:

mv libjcpuid.so libjcpuid.jnilib
java -Djava.library.path=. -jar lib/i2p.jar cpuid

[zzz]

Changes from comments 7 and 8 above in 3ae48955436c1f449933da804f47353e3e3f6dd7 0.9.29-2

[Aniba]

I continue to observe large CPU performance differences between shipped jbigi libraries and a natively compiled one. This is v. 0.9.31.

Running a decent bandwidth router that enables floodfill with the shipped libraries yields a java process that eats at least 50–80% CPU.

Using my own natively compiled gmp binaries eats 10% CPU.

Here's the steps I take:

There is no /usr/local/lib/libgmp.so dynamic library for macOS. The easiest way to get this is with Macports. These commands download, compile, and set Macports gcc as the default compiler if you put /opt/local/bin first in the PATH environment variable.

sudo port install gmp
sudo port select --list gcc
sudo port select --set gcc mp-gcc6
export PATH=/opt/local/bin:$PATH
gcc --version

Now replace /usr/local/lib with /opt/local/lib in the file core/c/jbigi/build_jbigi.sh.

$ diff build_jbigi.sh build_jbigi.sh.orig 
81c81
<         LIBPATH="-L.libs -L/opt/local/lib"
---
>         LIBPATH="-L.libs -L/usr/local/lib"

Now build the dynamic library:

./build.sh dynamic
sudo cp lib/libjbigi.jnilib /path/to/i2p

[zzz]

Closing fixed after testing and no adverse reports.

Re: comment 12, thanks for the info about how to build on a Mac. Glad you got one built that goes fast.