Script does not find target image when App is in fullscreen mode

Asked by Raj Subramanian on 2018-06-06

Hi,

I am working on GUI automation for a Ubunu application.
The script (.sikuli) is unable to find the target images when the App is in fullscreen mode.
The script works perfectly well when the App is NOT in full screen mode.
Appreciate any help.

Thanks,
Raj

Question information

Language:
English Edit question
Status:
Answered
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2018-06-07
Last reply:
2018-06-07
RaiMan (raimund-hocke) said : #1

Please check, wether the rendering (pixel width, height, ...) of the images shown on the screen in both modes is different.

RaiMan (raimund-hocke) said : #2

You might make the following test in fullscreen mode, to check, wether it works at all:

# test script
wait(5) # time to manually switch to fullscreen mode
hover(find(capture()))

this should allow you to capture the respective image, then find it and move the mouse to the center of the image.

RaiMan (raimund-hocke) said : #3

What exactly is your Ubuntu setup?
GUI, Java, SikuliX, ...

Raj Subramanian (subrar) said : #4

Hi Raiman,

Thanks for response. Please find additional info.

I tried your test script. It just captures only the ubuntu desktop background.
The application which is running in fullscreen mode is not captured by the script.
Same happens when i tried to take screenshot using the Sikuli IDE.
But when I run the application without fullscreen, both your script and IDE captures the application home screen correctly.
I have no idea why this happens, but it looks like the application home screen is not found in fullscreen, but only the ubuntu desktop.

And for your questions,
1. There is no difference in rendering the images, i mean pixel width and height.
2. This is my set up - Ubuntu 16.04, QT Application, Java version 1.8.0_171, Sikulix version 1.1.2

Appreciate your support.

Thanks,
Raj

Raj Subramanian (subrar) said : #5

Additional info:
I found that the application GUI is based on QT.
Fullscreen mode is set as a borderless window using the QT API “showFullScreen()”

Does this make any difference in sikuli identifying the application?

RaiMan (raimund-hocke) said : #6

In the background SikuliX uses the Java AWT Robot class to capture the screen content in any case.

So it seems, that the screen content (some internal buffer) does not contain the app content in fullscreen mode and hence is not accessible for SikuliX.

conclusion: not possible - no solution.

Can you help with this problem?

Provide an answer of your own, or ask Raj Subramanian for more information if necessary.

To post a message you must log in.