SikuliX

Is it possible with sikuli to take a screenshot, within the screenshoted image create a rectangle around the text then read that text or text?

Asked by Emmanuel Mbambo

Hi,

When I was developing my own sekuli automation using both sikulix IDE(I'm using IDE to take screenshots) and Java on eclipse, I find a slight problem.

When using a region to find and or read text, sometime the images you want to interact with on a website are little bit down on the screen and I would need to scroll down for sikuli to find the images. This offsets the region where I want to read the text.

For me the solution I found is to zoom-in a little bit for all images to appear at once.

With image patterns I can take a screenshot and use offset to click at a specific place on an image. Correct me if I'm wrong I think the pattern will look for an image on a screen and use the screenshoted image to look for a specific place to click on a screen, it does not use a region. Which I think is more reliable even when the website is scrolled up and down, as long as the image is visible the click function or other supported functions will be possible.

This got me thinking and to ask this question.

Is it possible with sikuli to take a screenshot, then within the screenshoted image create a rectangle around the text or around where the text will appear(if the text is on and off) then read that text as normal?

The idea is the text will be found even if the page has scrolled up or down as long as the image is visible.

Question information

Language:
English Edit question
Status:
Expired
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

if I understand right:

regionOfImage = find(someImage)

now you want to read the text in some smaller region inside the found image?

You can calculate the new region based on the (x, y) of the given region:

newX = regionOfImage.x + xOff
newY = regionOfImage.y + yOff
textRegion = Region(newX, newY, w, h)

xOff, yOff, w, h have to be given or calculated before.

There are also some functions to define a new region based on a given one:
https://sikulix-2014.readthedocs.io/en/latest/region.html#extend-regions-and-create-new-regions-based-on-existing-regions

Revision history for this message
Emmanuel Mbambo (emmasibu) said : 		#2

In java do you call regionOfImage as a pattern then find the region within the image methods available?

Pattern regionOfImage = new Pattern("someImage");
regionOfImage.similar((float) 0.7);

Revision history for this message
Launchpad Janitor (janitor) said : 		#4

This question was expired because it remained in the 'Open' state without activity for the last 15 days.