Chrome: recent changes --- most of my test-pictures no longer useable

Asked by Marc Summers on 2018-12-10

I am working with a Chrome embedded frame work, and very recently changes were made to Chrome
and then those changes were applied to the Chrome embedded framework.

It appears that the Tesseract is not seeing many of the pictures that
before the change, were working OK.

I can fix some of the pictures by changing the similar from high like 0.98 to 0.87
but some of the pictures have to be re-snap shot in order to get them to work.

This is a really big issue since I have over 800 pictures in my total tests sets,
Is there any global tweeks to the Tesseract that could be done in order to
resolve this ?

Thanks

Question information

Language:
English Edit question
Status:
Answered
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2018-12-11
Last reply:
2018-12-11
RaiMan (raimund-hocke) said : #1

--- It appears that the Tesseract is not seeing many of the picture
... I do not understand what this has to do with SikuliX?

Mike (maestro+++) said : #2

I know it doesn't help but I believe it is Open CV that 'sees the pictures'. Tesseract is used for OCR.

Your question however is very interesting. It can be re-phrased as "How can you protect yourself from changes to images?"

RaiMan (raimund-hocke) said : #3

If you are talking about SikuliX features (supposing you use them), then the complete image handling is done in Java with Java features.
OpenCV is used for image search (match-template) and Tesseract is used for OCR in images.

If you say:
images are no longer useable for search

... then this means, that images you see on the screen now and the older shot to search with no longer have the same pixel content.

So you have to analyze, what exactly has changed.

Take one example, make a new shot and compare it in an image viewer resized, so you can see the pixels.

Only if there is some systematic change, that could be compensated programmatically, then there is a chance.

Otherwise you have to recapture the images.

The FindFailed options to recapture the not found images while the script is running might help, to reduce the effort.

Mike (maestro+++) said : #4

I know we don't know what your applications involve but 800 pictures seems an awful lot. SikuliX examples tend to look for an icon or panel or box etc then look for another one and so on. If the target 'pages' are relatively stable you might instead trying to establish your self on a place on a page and then use positioning and/or tabs or other keystrokes to navigate around the page rather than keep looking for images.

Marc Summers (2aircraft) said : #5

Thanks for your answer RaiMan.

You speak of The FindFailed options to recapture the not found images while the script is running might help, to reduce the effort

Could you go into some more detail, or point me in the right
direction of the documentation.
I have not heard of this capability, or the syntax of how to implement it.
Any help would be much appreciated.

Thanks.

PS: Yes, this is what I am going to have to do:
images are no longer useable for search
... then this means, that images you see on the screen now
and the older shot to search with no longer have the same pixel content.
So you have to analyze, what exactly has changed.

masuo (masuo-ohara) said : #6

@Marc Summers

Document is here.
https://sikulix-2014.readthedocs.io/en/latest/region.html#Region.setFindFailedResponse

[example code1:]
click("image.png")

If "image.png" changed, FindFailed exception will occur at this step.

[example code2:]
setFindFailedResponse(PROMPT)
click("image.png")

In this case, you will be prompted for the way to handle the situation instead of FindFailed exception.
When you select Capture/Skip button, you would allow to recapture the image immediately.

Can you help with this problem?

Provide an answer of your own, or ask Marc Summers for more information if necessary.

To post a message you must log in.