macOS M1: how to use a custom build of OpenCV

Asked by Jeff Harmon

I am trying to get native Java working with Sikuli on an M1 Mac. With few options of Java built native to M1, I am trying out Java 11 from Zulu based on OpenJDK

openjdk version "11.0.11" 2021-04-20 LTS
OpenJDK Runtime Environment Zulu11.48+21-CA (build 11.0.11+9-LTS)
OpenJDK 64-Bit Server VM Zulu11.48+21-CA (build 11.0.11+9-LTS, mixed mode)

JVM seems to be working well enough, but because java is native it means the libopencv_java430.dylib library doesn't work since it was built for Intel Mac.

I have build opencv myself so that I have a native M1 version available, but I don't see how to tell Java to use my instance of the opencv library instead of what is in the sikulixapi-2.0.5.jar file. I built the most current version of opencv, so I have libopencv_java452.dylib along with opencv-452.jar from my native build.

I am pulling sikulixapi-2.0.5.jar into my project via maven dependency, so what is the best way to override the native libraries included in sikulixapi-2.0.5.jar and point to the native library I have built?

Question information

English Edit question
SikuliX Edit question
No assignee Edit question
Solved by:
Jeff Harmon
Last query:
Last reply:
Revision history for this message
Jeff Harmon (phototacopodcast) said :

I have made a sym link in Library/Application Support/Sikulix/SikulixLibs to my native build of opencv:

libopencv_java430.dylib -> /usr/local/share/java/opencv4/libopencv_java452.dylib

I get through initialization but then the first attempt to use the api I get an exception while it is trying to make the call through libMacUtil.dylib:

[27026 debug] RunTime:loadlib: trying MacUtil
[error] RunTime:loadLib: libMacUtil.dylib (failed) probably dependent libs missing:
/Users/jeffharmon/Library/Application Support/Sikulix/SikulixLibs/libMacUtil.dylib: dlopen(/Users/jeffharmon/Library/Application Support/Sikulix/SikulixLibs/libMacUtil.dylib, 1): no suitable image found. Did find:
 /Users/jeffharmon/Library/Application Support/Sikulix/SikulixLibs/libMacUtil.dylib: mach-o, but wrong architecture
 /Users/jeffharmon/Library/Application Support/Sikulix/SikulixLibs/libMacUtil.dylib: mach-o, but wrong architecture

Is there a way I can build libMacUtil.dylib to be native M1?

Revision history for this message
RaiMan (raimund-hocke) said :

I am sorry, but currently there is no prepared way, to do what you want.

In a few days I will have an iMac M1 available and will start to make 2.0.6 ready to run on such machines.

Until then, you can try the following:

1. exchange the dylib in ~/Application Support/Sikulix/SikulixLibs
make sure, that it gets the same name (libopencv_java430.dylib)

2. exchange the dylib in the jar
- unpack the jar
- find the dylib
- exchange with same name
- repack the jar

3. Maven project
Use the Maven features, to pack a final jar containing sikulixapi with the exchanged dylib

In all cases you will still have the Java OpenCV API of version 4.3.0. But since SikuliX only uses a few relatively stable features it might work so.

To just

Revision history for this message
Jeff Harmon (phototacopodcast) said :

I have the native opencv loading, but can't actually execute anything because the libMacUtil.dylib is still built for Intel.

I see the Support/nativeCode/mac/MacUtil.m file as the source to build that library, but I am struggling to build it.

Do you have a makefile or a config script for building that library?

Revision history for this message
RaiMan (raimund-hocke) said :

As mentioned: I am sorry, but I cannot help you in the moment. (hopefully within a week ;-)

I tried to find ways theoretically by digging in the net. But due to lack of an M1 system, I cannot check anything.

As far as I understand: In most cases x64 based apps should run on M1 with the support of Rosetta 2.

What was not so clear: Must the binary stuff be universal? IMHO: no, but non-universal is slower due to the translation process.

The MacUtil dylib is no longer there in 2.0.6.
... and no, I do not have a makefile available.

Revision history for this message
Jeff Harmon (phototacopodcast) said (last edit ):

Appreciate your time RaiMan!

Everything worked under Rosetta 2, but I am struggling to get an observer to scan a Region fast enough. Here is some sample code I have used to measure:

Settings.ObserveScanRate = 100F;

Region histogramRegion = new Region(2236,113,323,200) //323x200 region to observe

String histogramRegionAppearCode = histogramRegion.onAppear(blankHistogramDevelopPattern.exact(),
            new ObserverCallBack() {
                public void appeared(ObserveEvent event) {
                    if (logger.isDebugEnabled()) logger.debug("test: found blank histogram");

        logger.debug("Starting histogram observer...");
        wait(10000); //milliseconds
        logger.debug("Stopping histogram observer");

        if (histogramRegion.hasEvents()) {
  "Found " + histogramRegion.getObserver().getCount(histogramRegionAppearCode));

I get a total of 10 events in 10 seconds when the Region has an exact match the entire time the observer is running. The automation I am trying to create has the histogram region match the blank pattern for only 0.13 seconds. I was hoping that going native on Java/OpenCV would allow for faster scans.

I have now hacked a port together of all the components. I have everything working with native OpenCV, a native port of MacUtil (used XCode to help me migrate things and had to do some minor refactors), and Zulu native JDK 11. Unfortunately, the code above still only gives me a match about once per second. Maybe my code to test the time it takes for an observer to have an appeared event is flawed?

My end goal is to have an observer tell me that the application has had the histogram region be blank and then not blank within 0.3 seconds.

Revision history for this message
Jeff Harmon (phototacopodcast) said :

BTW, in order for JNI calls to work under the Zulu native JDK I had to update JNA to version 5.8.0


Just thought that may help you as you look into updating Sikuli to have official native support on M1. It took me forever to find that gem. Until I did that I got JNA UnsatisfiedLinkError messages.

Revision history for this message
RaiMan (raimund-hocke) said :

First of all: thanks for your efforts and the information about the results.

I will surely come back with the one or other question the next days and hope, that it is ok, when I contact you directly on your launchpad id.

About the observe case: I will try to setup a similar case and look into it.

Revision history for this message
RaiMan (raimund-hocke) said :

Ok, got it.

There is another option value:
Settings.RepeatWaitTime = 0 (set globally for all regions)

... the default is 1

So an
leads exactly to 1 observe per second.

my test:
Settings.RepeatWaitTime = 0
Settings.ObserveScanRate = 50;

reg = Region(238,166,361,308)
img = "img.png" # 167x230
pimg = Pattern(img).exact()


def handler(evt):
    global count
    count += 1

count = 0
running = 5.0
reg.onAppear(pimg, handler)
print count/running

gives 33 observes per second.

All the best.

Revision history for this message
RaiMan (raimund-hocke) said :

... and yes: this should be in the docs ;-)

Revision history for this message
RaiMan (raimund-hocke) said :
Revision history for this message
Jeff Harmon (phototacopodcast) said :

Excellent! Exactly what I was needing. I can get the information I need without an issue now. At some point I may test to see what the impact of Rosetta2 is on things but since I have native working I am sticking with it.

The documentation looks great.

Happy to answer any questions you have as you work on full support for Apple Silicon. The problem I did not solve for was a universal binary that would work with either M1 or Intel. The build I did of MacUtil was for the arm64 architecture only and linking the dynamic library so that either Intel or M1 is supported given the dependency on the JDK where the libawt.dylib is one or the other is a challenge I didn't solve for.

You mentioned not using MacUtility anymore though so maybe you won't have to deal with that.

Revision history for this message
RaiMan (raimund-hocke) said :

There now is a downloadable snapshot of 2.0.6 IDE available

plus some info: