Finding the region of an image based on a similar image

Asked by Sarah Robertson on 2020-06-17

So what I'm trying to do is find an image based on a similar image, and then read in the text of the found image. I'm using Jruby and Rspec in VS Code.

Basically the software I'm testing takes a screenshot of a user-specified area and then produces a "drop" of that area, which is labeled with the date and time that it was taken. I'm trying to write a test that ensures that the label is in a certain format.

I have an image of an old drop (from 2020-06-03), and right now what I'm doing is creating a region based on doing a find for that image, and then doing a find within that region for the old drop in order to get the label of the current drop, and then extracting the text from that current drop and doing checks from there.

Here's the relevant code:

(in a Regions file)

def drop
   Sikulix.Screen.find(CloudApp::Pattern.drop_label)
end

(in a Patterns file)

def drop_label
    Sikulix.Pattern('images/win/drop_label.png').similar(0.7)
end

(in the spec file)

    drop_text = CloudApp::Region.drop.find(CloudApp::Pattern.drop_label).text().split(" ")
    expect(CloudApp::Action.check_drop_label(drop_text)).to eq(true)

check_drop_label is the function that makes sure the label is valid and works fine, but it consistently returns the text from the old drop (2020-06-03). I can't figure out how to get the current drop. I really feel like this is possible to do, but I'm struggling with it.

Question information

Language:
English Edit question
Status:
Solved
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
2020-06-25
Last query:
2020-06-25
Last reply:
2020-06-25
Best RaiMan (raimund-hocke) said : #1

In the current 2.0.4 there is a problem with someMatch.text(), so that it uses the searched image (old drop) and not the region where it was found for OCR.

The workaround:
def drop
   Sikulix.Region(Sikulix.Screen.find(CloudApp::Pattern.drop_label))
end

... cast the Match to a Region object.

see: https://bugs.launchpad.net/sikuli/+bug/1883605

Sarah Robertson (seyrejane) said : #2

That worked!! Thank you so much!