sikulix 2.0.4 adb screenshot always starting at position 0,0 // how to search for images on android devices

Asked by dezimodnar on 2020-11-25


thanks for Sikuli, it is a great tool and i use it for years.

I started trying to work on an android device, using Sikuli 2.0.4 and the experimental adb support.

I can connect and take screenshots of the device screen fine but when i try to capture an image to re-use it for actions like clicks, the following happens:

* dialog opens asking if i want to capture on the android device
* when i select yes, the android screen is displayed in a popup window
* when i try to define a region of interest by clicking on the popup window, it will automatically drag from the top left corner (i assume its position 0,0) to the point i clicked. instead of letting me define the region of interest rectangle

if i instead save a screenshot of the device to my computer and then try to do a capture on it, it is also not working:

* dialog opens asking if i want to capture on the android device
* i select no but instead of being able to capture on my local device, a small blue stripe is saved as an image immediatelly

if i try to crop the saved screenshot to the region of interest and then provide it as in click("image.png") the image is not found on the android screen (although it is visible)

how could i go about that and define images to click and search for?

Question information

English Edit question
Sikuli Edit question
No assignee Edit question
Solved by:
Last query:
Last reply:
dezimodnar (dezimodnar) said : #1

The android device is a Samsung Galaxy Grand Prime on Android 5.0.2 if it matters

dezimodnar (dezimodnar) said : #2

I got it to work using a different image editor and cropping the screenshot taken from the android device to the region of interest now. Any way to make the image capture work though?

RaiMan (raimund-hocke) said : #3

Sorry, but Android support might have bugs and many missing features, since it is not under development since a few years.

... and in the moment it is not my priority. You have to take it as is.

dezimodnar (dezimodnar) said : #4

Any plans to implement the

 //TODO: get device by id

in anytime soon? Being able to attach multiple devices and defining which device to run the script on would be great

RaiMan (raimund-hocke) said : #5

I currently do not have any plans with the ADB support.

You might try to implement it yourself at the script level.
In Jython you can use available Java classes and their methods.

dezimodnar (dezimodnar) said : #6

Thanks good hint, am working on subclassing the java in my script now and adding methods that do what i want :) Can read the serialnumber of all attached devices from sikuli so far