SikuliX

Using OCR to select day of the month

Asked by Daniel Imperiale

I'm just getting started using Sikuli so this may seem pretty rudimentary so forgive me if that's the case.

I have managed to figure out how to find a region and parse the text and check for text within that region .
For example I use:
imageText = find("1577221295292.png").text()
print(imageText)
print("##################")
print(imageText.find("December"))
print(image.Textfind('January'))

This is the calendar in question: https://i.imgur.com/trqvUeB.
Using this script if the calendar shows December it prints a 0 and a -1 for January.

Now I'd like to be able to search WITHIN that image for the location of the number 1 -> 31 so that I can feed them into a second automation task that clicks the correct day of the month that is fed in through a script.

I assume I have to search within that image for the x/y coord and feed in images of the calendar dates. But I am struggling to understand what all the objects are that I'm interacting with and how to access their methods/properties.

thanks so much!

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Daniel Imperiale
Solved:
Last query:
Last reply:
Revision history for this message
TestMechanic (ndinev) said : 		#1

Here is one example that works for me on google calendar
https://www.screencast.com/t/M8bUxD8pHv2u

However there are some important notes:

-- OCR may not work well on your calendar
-- Extra logic need to be applied to handle repeating appearance of 30, 31 and 1-9 in my example
-- Alternative approach with images of all dates can be working as well

Revision history for this message
RaiMan (raimund-hocke) said : 		#2

I made some tests with your given calendar image.

I have to admit, that it seems, that the SikuliX internal optimisation of the images before doing the OCR still has to be improved further.

This is my test script, that you have to either test in the SikuliX IDE or transcript it to Java code.
Your calendar image is visible on the screen.

lnn = 5 # select a week
OCR = TextOCR.start()
reg = Region(163,89,228,198) # the calendar region
lines = 8 # number of lines in the grid
cols = 7 # number of columns
lh = reg.h/lines # line height calculated
cw = reg.w/cols # column width calculated
line0 = reg.above().below(lh)
#line0.highlight(1)
uprint(line0.text()) # prints December | 2019

# step through the weeks and days
for ln in range(lnn + 1, lnn + 2):
  line = Region(reg.x, reg.y + ln * lh + ln, reg.w, lh - 5)
  line.highlight(1)
  for cn in range(7):
    day = Region(line.x + cn * cw + cn + 2, line.y, cw - 5, line.h)
    #day.highlight(1)
    print OCR.doOCR(Image.create(capture(day)).resize(2)).strip()

--- comments
- the final line and day regions are adjusted in position and dimension, to cut out the grid frames
- the key point is to work with an image, that is resized with factor 2 (the simple method doOCR() with the standard optimisation does not deliver good results)

Revision history for this message
Daniel Imperiale (dimperiale) said : 		#3

Thank you so much for taking the time to both show me the google example as well as provding the code to get me started with a poc.
I still think it's workable but I may end up trying to get calendar date x/y coordinates through a python script and the datetime library.