Setting JyNI to work with Processing

[op-rt]

Dear @Stewori ,

Thank you for the clarifications provided on the Processing.py git thread.

I (and some other folks from the Processing community) would love to see JyNI working in Processing Python mode but are completely overwhelmed by the installation instructions. Would you mind give us a hand with this ?

Here's what I could do so far:

• looked for the jython.jar file on my computer. It lies in a Processing sub-folder called 'mode' with 2 other jar files, PythonMode.jar and guava-17.0.jar

• set a JRE_HOME variable pointing to the JRE installation path C:\Program Files (x86)\Java\jre1.8.0_211

This leaves me with 2 questions:

• Do I need to put JyNI.jar and JyNI.dll in this 'mode' sub-folder (with the other jar files) or somewhere inside the JRE installation, like in C:\Program Files (x86)\Java\jre1.8.0_211\lib for example ?
• How do I make these 2 files available to the Java classpath ? (That's the million dollar question for us)

Your help, even guidelines or a simple hint, would be greatly appreciated.

Sincerely,

solub

[Stewori]

I think you have to examine the launch process of processing. To clarify: I hadn't had a look at it at all.
Is it launched by a shell script or bat file (on WIndows) or something? Or by an executable? How would you make any other ordinary Java library available to it?
What you can also try: Search the whole repo for the text 'jython.jar' or 'mode/Jython.jar' or 'mode'. This might point you to the launcher. Or search for '-cp'. Is there something in the doc explaining how to make Java libraries available?

When launching from command line via java ... the -cp=... flag configures the classpath. See https://github.com/Stewori/JyNI#running-jyni

That said, keep in mind that JyNI is highly experimental. Using unsupported API crashes the JVM. Attempting to import an extension that doesn't work yet crashes the JVM. Be prepared for frequent full JVM crashes. Some NumPy core functionality works but it is not very exhaustive.

[Stewori]

You can try to insert additional classpath here and in processing-py.bat as well if you need it on WIndows. However, the setup is fairly complex and maybe it somehow overides the classpath configured there. It might require some serious effort to sort things out.

[op-rt]

Thank you so much for the reply.

All the libraries are available in the libraries sub-folder of Processing. Each library folder contains another folder called 'library' where the jar file is usually placed.

Processing -> libraries -> [LibraryName] -> library -> [LibraryName].jar

I already tried to create a JyNI folder inside libraries and put the jar file with the dll within a library sub-folder, following the same order. JyNI is then recognized as a library and I can import it in a Processing sketch however it is impossible to import numpy (1.13.3).

add_library('JyNI')

import sys
sys.path.append('C:\Python27\Lib\site-packages')

import numpy as np

processing.app.SketchException: import * does not work in this environment.
	at jycessing.mode.run.SketchRunner.convertPythonSketchError(SketchRunner.java:234)
	at jycessing.mode.run.SketchRunner.lambda$2(SketchRunner.java:119)
	at java.lang.Thread.run(Thread.java:748)

When running the same sketch but replacing numpy 1.13.3 in site-packages by version 1.16.4, the error message is different:

processing.app.SketchException: ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:
- If you're working with a numpy git repository, try `git clean -xdf`
  (removes all files not under version control) and rebuild numpy.
- If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
- If you have already reinstalled and that did not fix the problem, then:
  1. Check that you are using the Python you expect (you're using C:\Users\solub\Documents\Processing\modes\PythonMode\mode\bin\jython.exe),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.
  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues.  Please include details on:
     - how you installed Python
     - how you installed numpy
     - your operating system
     - whether or not you have multiple versions of Python installed
     - if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: DLL load failed: %1 n?est pas une application Win32 valide

	at jycessing.mode.run.SketchRunner.convertPythonSketchError(SketchRunner.java:234)
	at jycessing.mode.run.SketchRunner.lambda$2(SketchRunner.java:119)
	at java.lang.Thread.run(Thread.java:748)

[Stewori]

Don't try NumPy > 1.13 atm. It won't work, see #22. Stick to 13.3 specifically.
Then before you attempt to import NumPy, which has a lot of potential failure possibilities, try to import datetime and confirm that datetime.__doc__ reads "Fast implementation of the datetime type.". Much like in JyNIDatetimeTest. In that code, the line sys.path.insert(0, '/usr/lib/python2.7/lib-dynload') shouldn't be required any more as of alpha 5: https://github.com/Stewori/JyNI/releases/tag/v2.7-alpha.5 as datetime is builtin now. (It may be required for other extensions though).

If datetime.__doc__ is other than "Fast implementation of the datetime type.", Jython's original datetime was loaded, meaning Jython did not recognize JyNI for some reoson. This can likely be caused by the clash of two environments being too smart (processing vs JyNI). E.g. JyNI actively searches the classpath for its native libs although they would usually reside on the library path rather than on classpath. Maybe processing loads the jars from the libraries folder using a custom class loader and not by putting them on the classpath. That might prevent JyNI from finding its native libs. You can try to put JyNI's native libs on the library path rather than on the classpath (I'm not sure if that works on Windows). You can specify the library path via the JVM flag -Djava.library.path.

[Stewori]

You may want to have a look at https://github.com/Stewori/JyNI/blob/master/JyNI-Java/src/JyNI/JyNI.java#L114 to better understand the logic. Make sure to observe System.err output for clues. Check if processing hides this output (maybe it overrides System.err) and how it can be accessed in that case.