SikuliX

Strange behavior with Chrome and Flash webapp

Asked by Spencer Keller

I've got a weird one. I'm writing automated tests for an Adobe Flash based webapp and I have a strange problem and I'm wondering if anyone else has ever seen this and possibly has a work-a-round.

I have some 40 test cases written and they all work fine on both Firefox and on Chrome. I wrote a new test the other day and for some strange reason it fails only on Chrome. The test is interacting with a popup dialog (which many of my tests do) and it's checking to see if an image appears in the dialog. I've written a little framework around Sikuli that writes the image being sought and an image of the browser window to the log file when a failure occurs. The image being sought is very clearly present in the browser image. I have tried changing what is being sought to VERY obvious images but still Sikuli does not “see” it. What is even more odd is that I always try to reduce the size of the Region that Sikuli needs to search so my test finds a label on the dialog and then “grows” the Region to define the area to search and Sikuli successfully finds this label and grows the Region. I know this because I use the Region.highlight() to draw a box around the search Region. It's almost as if the search Region is blank inside. The other thing is that I use this approach all over my test code and it works just fine, just not in this case. Very weird! Oh and I have played with the “similar” setting with no impact.

I am running on Windows 7 and all of my tests are in Java using the Sikuli Java SDK library.

Anyone have any thoughts?

Code snippet:
      Region findRegion = {dialog region}.wait({label on dialog}).grow(0, 300, 5, 130);
      findRegion.hightlight(1);
      Pattern pattern = new Pattern({image being sought}).similar(0.98f);

      // The line below is where the failure occurs.
      Region tmp = findRegion.wait(pattern);

Thanks,
Spence

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Spencer Keller
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

supposing it is 1.1.0 and single-monitor.

already tried without the highlight() ?

timing problem?
Region tmp = findRegion.wait(pattern, 10);

--- you say:
I've written a little framework around Sikuli that writes the image being sought and an image of the browser window to the log file when a failure occurs.
How do you do that?
On FindFailed save a shot at that moment?
If yes, this is not exactly the situation at failure time.
Have a look at Region.getLastScreenImageFile(), which is the capture, that is used for the search.

Revision history for this message
Spencer Keller (spencer-keller) said : 		#2

Hi RaiMan,

Timing was my first thought too so I added a sleep before the findRegion.wait() with not impact. I just did as you suggested and added 45 seconds to the wait(45) again no impact. I have not used Region.getLastScreenImageFile() before but added it to my framework and looked at the file. The image being sought is there. This is the strangest thing I have seen while using Sikuli. It is making NO sense to me. I am at a total loss. I have some more tests I need to get written but when I have some time I will experiment some more and see if i can find anything. I of course will post what I find, here.

Spence

Revision history for this message
Spencer Keller (spencer-keller) said : 		#3

What will Sikuli do if it finds two matches within a Region? Won't it just return the first match it finds?

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

The implementation of wait, find exists and the implicit find in click and other functions return the match with the highest score and only this.

see:
http://sikulix-2014.readthedocs.io/en/latest/basicinfo.html#sikulix-how-does-it-find-images-on-the-screen

Revision history for this message
Spencer Keller (spencer-keller) said : 		#5

Hello,

I went back and looked at this today and I have figured out what is/was going on and it has NOTHING to do with SikuliX. The issue is with Chrome and it's use of it's own Flash player. In my tests, I'm looking for images of text on the screen and have to set the “similar” value quite high, in the 0.95/0.98 range. In this particular case Chrome rendered the text such that I needed a lower “similar” setting. So a word of advise if you're testing Adobe Flash browser apps. keep this in mind. Chrome renders Flash differently than other browsers that use the Adobe Flash plugin. One other thing, the SikuliX “Pattern Settings” tool was invaluable in finding this problem. I opened both a Firefox and Chrome browser to the same page and then used the “Pattern Settings” tool to see that different “similar” settings were needed for each browser.

Spence