Java - Python Shared Memory Communication

We have one program in Java and one in Python, and need to get them taking together in a ping-pong manner, each time exchanging an integer array of length 100,000, and taking ~ 0.1

Solution 1:

Go for a fast intermediary data server to assist in communication between them. Redis would do the trick. You'll need two data structures there:

1. a list (your list of 100,000 items). We'll call that my_project:list for reference.
2. a lock. This can just be a Redis string set to "Python" or "Java."

Then have the following interaction:

1. Both Python and Java poll the Redis lock. If it's equal to "Python", it's Python's turn. If "Java," it's Java's turn.
2. Whichever program's turn it is goes into work mode and does whatever it needs to my_project:list, then it sets the lock to the other program's turn.
3. Repeat indefinitely.

Solution 2:

You could try combining java and python in the same process using jep. The latest release added support for sharing memory between python and java using numpy ndarrays and java direct buffers. This would let you share the data without any copying which should give the best performance possible.