getting FindFailed error in headless mode

Asked by Karan Joshi on 2020-04-24

Hello, I am using Sikulixapi jar in selenium using java. Using Selenium WebDriver I am able to run perfectly without headless mode, but in case of headless mode, I am getting error as FindFailed.
I am taking screenshot of browser in headless mode and it seems exactly same as normal browser (evey GUI element are perfectly on their place).
Can anyone please help me ??
I am sorry if I am missing anything....

Question information

Language:
English Edit question
Status:
Answered
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Last query:
2020-05-14
Last reply:
2020-05-15

This question was reopened

RaiMan (raimund-hocke) said : #1

system? Java? SikuliX version?

What exactly is "headless mode" in your case?

Karan Joshi (karan-23) said : #2

Hi RaiMan, thanks for the response.
Following are the information, in case if you want more feel free to ask.

System :
Windows 10 pro
64-bit operating system, x-64 based processor

Java :
openjdk version "13.0.2" 2020-01-14
OpenJDK Runtime Environment AdoptOpenJDK (build 13.0.2+8)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 13.0.2+8, mixed mode, sharing)

Sikulix Version:
I am using Sikuli from maven dependency with following version
sikulixapi-2.0.4.jar

About headless mode :
- I want to open chrome browser and perform automation on citrix application.
- I am using selenium webdriver to open the browser and then with the help of sikuli,want to perform object detection followed by some particular action ( such as click, hover, type ).
- All this stuff is perfectly running when browser is open on monitor but I want this task to run in background ( browser will not open on monitor ).
- Selenium web driver provides a way to open browser in background.
- However, when i am trying to running in this scenario sikuli is not detecting object.
- I have taken screenshot while browser running in backgound and it seems exactly as normal browser ( every GUI element are visible ).
- I have tried this solution too (https://stackoverflow.com/questions/26032706/how-to-get-sikuli-working-in-headless-mode ) but it did not worked.
- So, can you please help me to find solution, cause I have explored everything about it but unable to find any solution which worked.

Karan Joshi (karan-23) said : #3

Hello RaiMan, above issue can be solved if it is possible to configure sikulix ( object of Screen class ) with selenium webdriver.

This is my code without headless mode which is executing without any sort of error ( Browser is visibile ) :
                Screen s = new Screen();
  System.setProperty("webdriver.gecko.driver","geckodriver");
  FirefoxDriver driver = new FirefoxDriver();
                driver.get(URL);
                // then performing some action using sikuli ( such as detect object then click on )

When it comes to headless mode, code slightly changes as we need to pass arguments (This makes driver to open browser in background):

                Screen s = new Screen();
  System.setProperty("webdriver.gecko.driver","geckodriver");
                FirefoxOptions options = new FirefoxOptions();
  options.addArguments("--headless");
  FirefoxDriver driver = new FirefoxDriver(options);
                driver.get(URL);
                // then performing some action using sikuli ( such as detect object then click on )

In the second scenario, browser is running in background(i.e. We can't see it on screen ) but can be accessed by selenium web driver.

So, is it possible to configure sikuli with selenium web driver?? Cause, I did not get any way to configure it.
If it is, then it can help browser to execute in headless mode.
Thank you

RaiMan (raimund-hocke) said : #4

Only if you can see the website on the screen, you can act on it with the means of SikuliX.

As I understand, it is possible to create a screenshot using Selenium features in headless mode.
You could use this shot, to find your target in it and calculate the click location.
Finally the click has to be done with a Selenium feature.

Karan Joshi (karan-23) said : #5

Hi RaiMan, Thanks for the response !

RaiMan : As I understand, it is possible to create a screenshot using Selenium features in headless mode.
Karan : As per your above statement, I have taken Screenshot and I have image of target object whose location I want.

RaiMan : You could use this shot, to find your target in it and calculate the click location.
Karan : In headless mode, I am unable to find location of target object. Can you please suggest, how to get location of target object?
(With the help of screenshot and image of target object, is it possible to get location without opening any window using sikulix (headless)??)

RaiMan (raimund-hocke) said : #6

f = Finder(shot)
match = f.find(image)

upper left corner of shot is (0,0)

Karan Joshi (karan-23) said : #7

Thanks RaiMan, that solved my question.

Karan Joshi (karan-23) said : #8

Hi RaiMan, as in one of the above conversation you said to use selenium click feature.
But, this is not going to help in my case cause I want to automate citrix application which is running in background.
And selenium click feature does not work in citrix application.
I have tried Java Robot class too but it didn't work in background.

So, do you know any other way to click in an application at a particular location ( Application in running in background ) ??

Thanks

RaiMan (raimund-hocke) said : #9

Definitely not with Sikulix nor Java Robot.

Karan Joshi (karan-23) said : #10

Hi RaiMan, I am able to get location of element from screenshot .
But the location I am getting from the screenshot is different from the actual location of element on browser.
Actual location : (8,8)
Location getting from screenshot : ( 82,33 )

Can you please help me, how to solve this issue ?

RaiMan (raimund-hocke) said : #11

I guess the location in the shot is screen related (top left corner of screen is (0,0)) and the element location is relative to the top left corner of the browser content area.

So you have to do some calculation based on the Position of the browser window.

Can you help with this problem?

Provide an answer of your own, or ask Karan Joshi for more information if necessary.

To post a message you must log in.