[2.0.x] Mac: screen captures with Retina resolution --- not possible currently

Asked by diego on 2019-12-22

I am using the following code to capture a screenshot of a region and save it as an image:

reg = Region(Region(277,333,129,128))
img = capture(reg)
shutil.move(img, "/Users/diego/Desktop/test1.png")

Then I pass the image to Tesseract, an OCR engine.

Here's the issue: When I use a screenshot taken from Sikuli, Tesseract fails to recognize the text. When I take a screenshot of the exact same area using MacOS CMD+SHIFT+4 Tesseract recognizes the text. It seems like the capture from Sikuli is downsized 50%. How can I correct this?

Another detail: I need to perform the whole operation as fast as possible. I can't afford to get Sikuli to manually enter CMD+SHIFT+4 and take the screenshot like a human would.


Question information

English Edit question
Sikuli Edit question
No assignee Edit question
Solved by:
Last query:
Last reply:
RaiMan (raimund-hocke) said : #1

have you already tried with the latest Sikulix 2.0.1 and its text/OCR feature (based on Tesseract 4)?
If it works, it is surely faster than your file based solution.

text = Region(277,333,129,128).text()

Apparently you are working on a Retina display. On such displays, each image pixel is blown up to a square of 4 display pixels.
Mac's capture tool keeps this in the image, so such images are double in width and height.

SikuliX generally works with the original image pixels and hence such externally captured images have to be downsized before use (a feature contained in SikuliX).
To automatically adjust captured images on Retina displays is a feature in the newer Java versions 8+.

In your case there is no solution with older Sikuli(X) versions.

So you should upgrade to SikuliX 2.0.1 anyways ;-)

diego (adrian-veidt) said : #2

I still need to capture the high res images even if Tesseract is built-in into Sikuli. I have other reasons. How do I get around the low res captures even in the newer Sikuli?

Best RaiMan (raimund-hocke) said : #3

SikuliX generally works with the you-call-it-low-res images.
The pixel-blow-up of Retina or HIRes on Windows is only for humans starring at the screen ;-)

If really needed, the you have to capture your images outside.
For those you want to use inside SikuliX, you have to apply the resize feature:
- generally: Settings.resize
- per image: Pattern.resize()

diego (adrian-veidt) said : #4

I ran tests with the .text() feature and compared with running Tesseract on an image taken using the screencapture command. There really is a significant difference.

RaiMan (raimund-hocke) said : #5


I would appreciate, to get at least one example, where you had significant differences.

Thank you

Jeff_Vallis (vallis-pj) said : #6

1.I am on a Retina Display
2.Image Captured by Sikuli - is OK
3.copy of Image Captured by Sikuli - is not OK

sounds like its not just the IMac Capture Tool - Copy changes image as well

if you agree please mark solved