Sikuli 1.0.1 - find() identifies wrong text

Asked by Pathiyil

Hi,
I am using Sikuli 1.0.1 on a Windows machine. I have a simple script that tried to identify the cell in my application which has a specific date. The results I am seeing are very inconsistent - sometimes it does not identify the text. At times it even selects the wrong text. I re-ran the test with the same data in an excel sheet and the behavior was the same. Are there any tuning parameters for text search that might help with this ?

My script:

app = App.focus("Microsoft Excel")
region = App.focusedWindow()
cell = region.find("11/24/15")
doubleClick(cell)

With this script and the data below (in a single column in an excel sheet), it was selecting a cell with "11/26/18" a good number of times.

XYZ 4.150 06/10/14
XYZ 4.625 09/08/14
XYZ 5.375 01/30/15
XYZ 5.375 01/30/15
XYZ 5.125 04/10/15
XYZ 5.000 09/15/15
XYZ 3.375 11/24/15
XYZ 6.250 01/15/16
XYZ 5.750 03/15/16
XYZ 2.875 03/16/16
XYZ 4.750 06/14/16
XYZ 5.625 02/27/17
XYZ 5.625 02/27/17
XYZ 1.625 03/20/17
XYZ 6.500 07/31/17
XYZ 1.250 09/26/17
XYZ 5.375 12/05/17
XYZ 5.375 12/05/17
XYZ 1.500 02/19/18
XYZ 5.000 06/04/18
XYZ 4.625 07/15/18
XYZ 8.125 11/26/18
XYZ 8.125 11/26/18
XYZ 5.450 06/10/19
XYZ 4.375 03/16/21
XYZ 4.650 01/20/22

Thanks.

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

sorry, currently know. we have to live with it.

do you have a chance, to run the sheet with a larger font?

But there is the package xlrd, that you can use to access an XLS directly from a Sikuli script.

Revision history for this message
Pathiyil (pathiyil) said :
#2

I ran it with zoom set to 150%, but the issue did not go away - still it selects the cell with "11/26/18" most of the time.

I was running the excel test to isolate fonts etc. used in my application. The real test would have to work with the application, so unfortunately xlrd package wouldn't help me in this instance.

Revision history for this message
RaiMan (raimund-hocke) said :
#3

Ok, then there is no chance currently, to get that working with Sikuli for now in a robust way.

Another way would be to turn it around and try to use the OCR feature (Region.text()), to find out, what Sikuli reads in specific cell.

For the planned app testing this approach could be used if it works better.

Revision history for this message
Pathiyil (pathiyil) said :
#4

Tried to see what OCR gives in this case, by 'iterating' over the rows using Region.below(x).

cell_region = region.find("<Image of the column header>.png").below(20)

#Loop over the rows, read the text in the cell and print it out
for x in xrange(0, 25):
    print cell_region.text()
    cell_region = cell_region.below(20)
    click(cell_region)

The output is given below, which has a few interesting things:
  - For some reason, the beginning part of the string comes at the end (and seemingly with a new line prefixed to it)
  - Some existing spaces are getting stripped while some new spaces are getting inserted

The above 2 points might make even the comparison with Region.text() work in an inconsistent fashion. Ofcourse, Region.text() can be replaced with 'Copy and get the data from clipboard', but that might not work in all cases (especially with Read-only UIs).

Output of the script snippet with the input data specified in the original question:

4.15006/ 10/ 14
XYZ

4.62509/08/14
XYZ

5.37501/30/15
XYZ

5.37501/30/15
XYZ

5.12504/10/15
XYZ

5.00009/15/15
XYZ

3.37511/24/15
XYZ

6.25001/15/16
XYZ

5.75003/15/16
XYZ

2.87503/ 16/ 16
XYZ

4.75006/ 14/ 16
XYZ

5.62502/27/17
XYZ

5.62502/27/17
XYZ

1.62503/20/17
XYZ

6.50007/31/17
XYZ

1.25009/26/17
XYZ

5.37512/05/ 17
XYZ

5.37512/05/ 17
XYZ

1.50002/ 19/ 18
XYZ

5.00006/04/18
XYZ

4.62507/ 15/ 18
XYZ

8.12511/26/18
XYZ

8.12511/26/18
XYZ

5.45006/ 10/ 19
XYZ

4.37503/ 16/ 21
XYZ

Revision history for this message
Eugene S (shragovich) said :
#5

If you see that the recognition error is consistent you can attempt to perform some "post processing".

So for example if you consistently getting an unneeded space or new line, you can check that and make some corrections.

Can you help with this problem?

Provide an answer of your own, or ask Pathiyil for more information if necessary.

To post a message you must log in.