How to get Sikuli to identify images on a larger monitor

Asked by Ali Balkot

I have my Sikuli script working just fine - however when I run the same scripts on a different monitor size Sikuli can no longer identify the images.

Is there a workaround for this? Because it would take far too long to change all my scripts to accommodate the new screen size.

Thanks!

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

read this to understand:
http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

the other screen must be run with the same resolution as the one, the images where captured (same size in pixels).

Revision history for this message
Cam (ccreynolds2015) said :
#2

As RaiMan's post is correct, depending on what you are having the script to do, I may have some ideas. Are you using the script on a web browser or an application? If an application I would have no idea. But in a web browser you have a few options.

Revision history for this message
Ali Balkot (alibalkot3) said :
#3

Thanks RaiMan - a slight workaround I used was to just always set the browser size to be constant, so the images are always the same pixel size.

Hi Cam, thanks for getting back to me - I am using the script on a web browser and I'd love to hear your options? :)

Revision history for this message
Cam (ccreynolds2015) said :
#4

Alright, so I use an extension for Chrome called "Vimium", the Firefox counterpart is called "VimFX"

https://chrome.google.com/webstore/detail/vimium/dbepggeogbaibhgnhhndojpepiihcmeb?utm_source=gmail

Extension store link (free)^

What Vimium does is give you hotkeys for links/buttons on a screen. Now with it installed you can press "f" on a page and 1-2 digits will pop up for every link on the screen.

On the top right of your browser you should see a circle with the letter "V" in it. Click it, go to options, and ensure "Use the link's name and numbers for link-hint filtering" and "Use smooth scrolling" are both checked, and click save changes at the bottom of the page.

Return to the page you were working with. Try pressing "f" again. It will appear the same with the digits, however if you start typing the name of the link on the screen, it will start filtering them until no other options are left.

I.E/

>Uninstall Websphere
>Uninstall Word
>Uninstall Fishing Game

Notice how these share the same word, and two of them start with the letter "W" in the second word.

If you were to press "f" and type "Uninstall W" it would only display the first two options. Typing "Uninstall We" would immediately go to the "Uninstall Websphere" link as no other options can be eliminated.

Why is this useful? Hotkeys are universal for the most part where screen size isn't always. If you get the script you are working with to take advantage of these hotkeys, you will have a super easy time writing stuff up.

I know this looks like a lot, but it is really super easy once you have it :)

Revision history for this message
RaiMan (raimund-hocke) said :
#5

@Ali Balkot
thanks for the additional information.
Yes, webpages, that scale images in their content depending on the browser's window size are problematic for SikuliX.

What you did is one option.

Another solution might be, to somehow get the current scaling factor (e.g. with Selenium or other tools, that can get the image size somehow) and then scale the images before using them to search for.
This is possible with 1.1.1 with some investment in functions to do that.

I have this now on the list for version 2: an option to apply a scaling factor to the images before they are used for search.

Revision history for this message
Ali Balkot (alibalkot3) said :
#6

Hi,

My apologies for the delayed response.

Cam, although that sounds like a great idea, unfortunately this would not work for my scenario as I am working on a HTML5 Canvas application, which means there are no DOM elements to select/manipulate - it is simply one image/palette.

Thanks Raiman I'll try see if I can use a scaling factor to scale the images and see how I get on - would be really good if a scaling factor for images could be implemented in another release in Sikuli. :)

Can you help with this problem?

Provide an answer of your own, or ask Ali Balkot for more information if necessary.

To post a message you must log in.