Java - Finder unable to find target image

Asked by Jai P on 2017-07-31

I am combining Appium and Sikuli to be able to automated my native app on iPhone by following the methodology discussed here - https://medium.com/@SimonKaz/appium-with-image-recognition-17a92abaa23d. Specifically I am reusing the OCR.java here - https://github.com/Simon-Kaz/AppiumFindByImage/blob/master/src/main/java/OCR.java

The modified getCoords() method I am using is below -

 public Point2D getCoords(BufferedImage baseImg, String targetImgPath, double minSimilarityValue) {
  // set new minimum similarity
  Settings.MinSimilarity = minSimilarityValue;
  Match m;
  Finder f = new Finder(baseImg);
  Point2D coords = new Point2D.Double(-1, -1);

  f.find(new Pattern(targetImgPath).similar((float) Settings.MinSimilarity));
  if (f.hasNext()) {
   m = f.next();
   coords.setLocation(m.getTarget().getX(), m.getTarget().getY());
  }
  // revert to default similarity
  Settings.MinSimilarity = DEFAULT_MIN_SIMILARITY;
  return coords;
 }

This doesn't find my target pattern (subset image) regardless of the similarity. It does work when I set the target to be the same as the base image. Here are the logs -

[debug] Debug.init: from sikuli.Debug: on: 3
[debug] RunTimeINIT: loadOptions: check: /Users/JP/ppium_mobile
[debug] RunTimeINIT: loadOptions: check: /Users/JP
[debug] RunTimeINIT: loadOptions: check: /Users/JP/Library/Application Support/Sikulix/SikulixStore
[debug] RunTimeINIT: global init: entering as: API
[debug] RunTimeINIT: ScreenDevice 0 has (0,0) --- will be primary Screen(0)
[debug] RunTimeINIT: Monitor 0: (0, 0) 1920 x 1200
[debug] RunTimeINIT: runs as sikulixapi.jar in: /Users/JP/sikulix
[debug] RunTimeINIT: exists libs folder at: /Users/JP/Library/Application Support/Sikulix/SikulixLibs_201703300100
***** show environment for API (build 201703300100)
user.home: /Users/JP
user.dir (work dir): /Users/JP/appium_mobile
user.name: JP
java.io.tmpdir: /var/folders/f0/stp5z28967v65lqf8yxdfcxm0000gn/T
running 64Bit on Mac OSX (10.12.5) from a jar
java 8-64 version 1.8.0_60-b27 vm 25.60-b23 class 52.0 arch x86_64
app data folder: /Users/JP/Library/Application Support/Sikulix
libs folder: /Users/JP/Library/Application Support/Sikulix/SikulixLibs_201703300100
executing jar: /Users/JP/sikulix/sikulixapi.jar
*** classpath dump sikulix
 46: /Users/JP/sikulix/sikulixapi.jar
*** classpath dump end
***** show environment end
[debug] RunTimeAPI: global init: leaving
[debug] RunTimeAPI: initAPI: entering
[debug] RunTimeAPI: resourceList: enter
[debug] RunTimeAPI: resourceLocation: (class org.sikuli.script.RunTime) /Lib/sikuli
[debug] RunTimeAPI: resourceList: having jar: jar:file:/Users/JP/sikulix/sikulixapi.jar!/Lib/sikuli
[debug] RunTimeAPI: files exported: 7 from: Lib/sikuli to:
 /Users/JP/Library/Application Support/Sikulix/Lib/sikuli
[debug] RunTimeAPI: initAPI: leaving
[debug] RunTimeAPI: loadLib: libVisionProxy.dylib
[debug] Image: loaded: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: cached: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (103 KB) (# 1 KB 103 -- 0 % of 64 MB)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
[debug] Image: reused: /Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png (file:/Users/JP/appium_mobile/src/test/resources/DelMonteOffer.png)
FAILED: chooseStackAndSignup
org.openqa.selenium.TimeoutException: Expected condition failed: waiting for ss.qa.utilities.OCR$1@1654a892 (tried for 15 second(s) with 500 MILLISECONDS interval)
Build info: version: '3.4.0', revision: 'unknown', time: 'unknown'
System info: host: 'JaiPadmbhansMBP.home', ip: '192.168.1.7', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.12.5', java.version: '1.8.0_60'
Driver info: io.appium.java_client.ios.IOSDriver
Capabilities [{app=/Users/JP/Downloads/SS-4033.ipa, networkConnectionEnabled=false, noReset=true, appiumVersion=1.6.5, databaseEnabled=false, deviceName=iPhone 6s, platform=MAC, platformVersion=10.3.3, webStorageEnabled=false, locationContextEnabled=false, automationName=XCuiTest, browserName=, takesScreenshot=true, javascriptEnabled=true, platformName=iOS, udid=d31fcf94b28417b74bed1c71383da888c7de6143}]
Session ID: ac5a90f1-a238-4115-ba36-11609d7e88a8

Question information

Language:
English Edit question
Status:
Expired
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2017-07-31
Last reply:
2017-08-16

This question was reopened

Jai P (jaip) said : #1

I used the grab.app on Mac and was able to get SikuliX to match the target image. Cmd+Shift+4 or Apowersoft Mac Screenshot did not work for me. I am on macOS Sierra 10.12.5

Jai P (jaip) said : #2

I used the grab.app on Mac and was able to get SikuliX to match the target image. Cmd+Shift+4 or Apowersoft Mac Screenshot did not work for me. I am on macOS Sierra 10.12.5

dinev (ndinev) said : #3

This is not answering your question but might be helpful in your case:

NativeScript's Open Source Functional Testing Framework
https://www.nativescript.org/blog/nativescript-open-source-functional-testing-framework

Launchpad Janitor (janitor) said : #4

This question was expired because it remained in the 'Open' state without activity for the last 15 days.