[1.0.1] Java: Pattern(BufferedImage) is not found (silently does nothing) --- usage: load images from jar

Bug #1220640 reported by RaiMan
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
SikuliX
Fix Committed
Critical
RaiMan

Bug Description

Hello,

I am a relative newcomer to Java coding and Sikuli both so please keep that in mind as you try to guide me on my way! Also, before coming here, I searched through existing questions on this site and many others, but have been unable to find something that explains a solution in a way that I understand.

Using Eclipse, I am attempting to write stand alone executable in Java that searches for images and clicks on them using org.sikuli.script classes but I cannot seem to get either an executable jar or exe version of my code to match images on-screen. In Eclipse, when I create a pattern using relative path names the script matches the image on-screen. For example, if I do:

Pattern imagePattern = new Pattern("src/Images/PizzleField.PNG");
or
Pattern imagePattern = new Pattern("bin/Images/PizzleField.PNG");

Then run my script within Eclipse, the script will find the above image without a problem. But, if I right click my project and Export as a runnable JAR file then attempt to run that JAR file, the image is no longer found. If I unzip the executable JAR, I can see that the Images directory with all of the images is there so the image are definitely being added to the JAR file. My research has led to look at using something along the lines of:

this.getClass().getClassLoader().getResource("Images/PizzleField.PNG");

to handle resources that are stored in a JAR file. Since it looks like I can create a Pattern using a BufferedImage, I have tried to use that to access and manipulate the images in my executable JAR file. Here is the code I am currently toying with to try and get this to work in Eclipse:

public boolean test() throws Exception {

  URL imageURL = this.getClass().getClassLoader().getResource("Images/PizzleField.PNG");
  String imageString = imageURL.toString();
  BufferedImage imageBuffered = ImageIO.read(imageURL);

  System.out.println("imageURL is: " + imageURL);
  System.out.println("imageString is: " + imageString);
  System.out.println("imageBuffered is: " + imageBuffered);

  Pattern imagePattern = new Pattern(imageBuffered); //("bin/Images/PizzleField.PNG");

  Screen newScreen = new Screen();
  float floatPercentage = convertIntMatchToFloatMatch(80);
  System.out.println("=====Searching for image: " + imageBuffered); //"bin/Images/PizzleField.PNG");
  if(newScreen.exists(imagePattern.similar(floatPercentage), 3) != null){
   System.out.println("+++++Found image : " + imagePattern);
  }
  else{
   System.out.println("ERROR: Could not find an onscreen match for the following image ----> " + imagePattern);
  }
}

When I run that code in Eclipse, the output I get is:

imageURL is: file:/C:/eclipseprojects/ffauto/ffauto/bin/Images/PizzleField.PNG
imageString is: file:/C:/eclipseprojects/ffauto/ffauto/bin/Images/PizzleField.PNG
imageBuffered is: BufferedImage@7646bb9f: type = 6 ColorModel: #pixelBits = 32 numComponents = 4 color space = java.awt.color.ICC_ColorSpace@1dc80063 transparency = 3 has alpha = true isAlphaPre = false ByteInterleavedRaster: width = 326 height = 22 #numDataElements 4 dataOff[0] = 3
=====Searching for image: BufferedImage@7646bb9f: type = 6 ColorModel: #pixelBits = 32 numComponents = 4 color space = java.awt.color.ICC_ColorSpace@1dc80063 transparency = 3 has alpha = true isAlphaPre = false ByteInterleavedRaster: width = 326 height = 22 #numDataElements 4 dataOff[0] = 3
ERROR: Could not find an onscreen match for the following image ----> P(-- BufferedImage --) S: 0.8

It looks like the imagePattern using the buffered image displays the contents of the image in the console log, but it never finds a match. If I change:

Pattern imagePattern = new Pattern(imageBuffered);
to
Pattern imagePattern = new Pattern("bin/Images/PizzleField.PNG");

so that the pattern is based on a String instead of a BufferedImage, then a match is found on-screen.

What am I doing wrong in the code above that is preventing my BufferedImage based Pattern to fail to find a match?

My Project Heirarchy is:

FFAuto
|
|-------src
| |
| |-------(default package)
| |
| |-------Images

Thanks in advance for any help. If any additional details are needed please let me know!

RaiMan (raimund-hocke)
Changed in sikuli:
status: New → Incomplete
status: Incomplete → In Progress
importance: Undecided → High
assignee: nobody → RaiMan (raimund-hocke)
milestone: none → 1.1.0
Revision history for this message
RaiMan (raimund-hocke) wrote :

I have already implemented getting images from a jar and use them with find() as buffered images and other features that internally use find() (findAll, observe and some minor aspects have to be adapted still).
The usage is rather convenient, since with one setImagePath() you can say, which jar on the classpath to use by just giving a classname and for the cases you do not run from the jar (e.g. when running/debugging in the IDE) you can give an alternate file path to load the images from.
so a find("images/some_image.png") will work in both cases.

I am a little bit behind, but hope I can give you something to test latest on monday.

RaiMan (raimund-hocke)
Changed in sikuli:
status: In Progress → Fix Committed
Revision history for this message
xsli2 (xsli2) wrote :

I need this feature too. I am not clear what new feature you have added. Based on your post, I would thought the code below will work:

Match match = screen.find("/images/abc.png"); //i also tried screen.find("images/abc.png")

if(match != null)
{
     System.out.println("I found the image");
}
else
{
    System.out.println("I could not find the image")
}

However, I got the error:
[error] /images/abc.png looks like a file, but not on disk. Assume it's text.
FindFailed: Text search currently switched off
  Line 1671, in file Region.java
[error] Region.find(text): text search is currently switched off

Thank you

Revision history for this message
RaiMan (raimund-hocke) wrote :

@xsli2
loading an image from a jar or alternatively from somewhere else (when running in an IDE) should work with 1.1.0-Beta1+

usage:
https://dl.dropboxusercontent.com/u/42895525/docs/index.html
ImagePath.add
the 2nd parameter can be a path (relative to working folder or absolute), that is used if not running from Jar (not needed for Maven projects)

RaiMan (raimund-hocke)
Changed in sikuli:
importance: High → Critical
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.