Using OCR to select day of the month

Asked by Daniel Imperiale on 2019-12-24

I'm just getting started using Sikuli so this may seem pretty rudimentary so forgive me if that's the case.

I have managed to figure out how to find a region and parse the text and check for text within that region .
For example I use:
imageText = find("1577221295292.png").text()

This is the calendar in question:
Using this script if the calendar shows December it prints a 0 and a -1 for January.

Now I'd like to be able to search WITHIN that image for the location of the number 1 -> 31 so that I can feed them into a second automation task that clicks the correct day of the month that is fed in through a script.

I assume I have to search within that image for the x/y coord and feed in images of the calendar dates. But I am struggling to understand what all the objects are that I'm interacting with and how to access their methods/properties.

thanks so much!

Question information

English Edit question
Sikuli Edit question
No assignee Edit question
Solved by:
Daniel Imperiale
Last query:
Last reply:
TestMechanic (ndinev) said : #1

Here is one example that works for me on google calendar

However there are some important notes:

-- OCR may not work well on your calendar
-- Extra logic need to be applied to handle repeating appearance of 30, 31 and 1-9 in my example
-- Alternative approach with images of all dates can be working as well

RaiMan (raimund-hocke) said : #2

I made some tests with your given calendar image.

I have to admit, that it seems, that the SikuliX internal optimisation of the images before doing the OCR still has to be improved further.

This is my test script, that you have to either test in the SikuliX IDE or transcript it to Java code.
Your calendar image is visible on the screen.

lnn = 5 # select a week
OCR = TextOCR.start()
reg = Region(163,89,228,198) # the calendar region
lines = 8 # number of lines in the grid
cols = 7 # number of columns
lh = reg.h/lines # line height calculated
cw = reg.w/cols # column width calculated
line0 = reg.above().below(lh)
uprint(line0.text()) # prints December | 2019

# step through the weeks and days
for ln in range(lnn + 1, lnn + 2):
  line = Region(reg.x, reg.y + ln * lh + ln, reg.w, lh - 5)
  for cn in range(7):
    day = Region(line.x + cn * cw + cn + 2, line.y, cw - 5, line.h)
    print OCR.doOCR(Image.create(capture(day)).resize(2)).strip()

--- comments
- the final line and day regions are adjusted in position and dimension, to cut out the grid frames
- the key point is to work with an image, that is resized with factor 2 (the simple method doOCR() with the standard optimisation does not deliver good results)

Daniel Imperiale (dimperiale) said : #3

Thank you so much for taking the time to both show me the google example as well as provding the code to get me started with a poc.
I still think it's workable but I may end up trying to get calendar date x/y coordinates through a python script and the datetime library.