[How To] set variable value from OCR numerics

Asked by Phillip Kelly

I have limited knowledge in programming Sikuli but it seems like a powerful tool. Is it possible to have the OCR read a numeric value such as a roulette table account balance and update a corresponding variable as the account balance changes ?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

This question was reopened

Revision history for this message
RaiMan (raimund-hocke) said :
#1

you have to check wether Sikuli can "read" the given font.

easiest way to check in IDE:

# take care that the account balance is visible on screen
print "the number is:", selectRegion().text()

This lets you select the account balance and tries to "read" the numbers. Watch what is shown in the message area.

It either works or does not work. In the latter case you can do nothing, but try to mark the optimal region.

(with version 1.0.0 you have to switch on OCR in the preferences -> more options)

Revision history for this message
Phillip Kelly (pjk-1966) said :
#2

thanks Raiman that helped alot. I managed to store the text in a variable by using:

balance = find().text()
which works fine except for one little detail. the resulting text includes a 3 character prefix which I dont want such as USD3000.00

How can I truncate this text to remove the "USD" and leave me with only the numeric value ?
In addition, How can I ensure the resulting text is actually numeric such that it can be manipulated for calculations using simple math ?

Revision history for this message
RaiMan (raimund-hocke) said :
#3

# using regular expression matching

import re

m = re.match("[^0-9]*([0-9]*\.?[0-9]*)", balance)
try:
    val = m.group(1)
except:
    val = -1
print val

Revision history for this message
Phillip Kelly (pjk-1966) said :
#4

the code seems to have a syntax error

 m = re.match("[^0-9]*([0-9]*\.?[0-9]*)", balance)

[error] SyntaxError ( "mismatched input '' expecting EOF", )

Revision history for this message
Phillip Kelly (pjk-1966) said :
#5

Thanks RaiMan, that solved my question.

Revision history for this message
Phillip Kelly (pjk-1966) said :
#6

I have another small problem because I don't fully understand how to use global and local variables properly.
When I try to pass the decoded balance to a global variable it fails and remains defined by the local function variable.

val = 0
result = 0

def readbalance( balance ):
    import re
    m = re.match("[^0-9]*([0-9]*\,?[0-9]*\.?[0-9]*)", balance)
    val = m.group(1)
    result = val
    print val
    print result
    return

Although I have defined both val and result as gobal variables, the function definition defines additional local variables with similar names.
How do I pass the decoded local variable "val" to a global variable such as "result" ?

Revision history for this message
RaiMan (raimund-hocke) said :
#7

as a convention and saving some computing resources: all imports should be at the beginning of your script

This would be a (recommended ;-) more generalized version of the function and it returns a number:

def getValue( text ):
    m = re.match("[^0-9]*([0-9]*\.?[0-9]*)", text)
    try:
        return m.group(1) # returns the found value
    except:
        return -1 # returns -1 if any problems

usage:
balance = "USD30000.00" # as read by the previous region.text() function
bval = getValue(balance)
print "balance-text:", balance, "balance-value", bval

your case even shorter:
balance = getValue( find().text() )
# now balance already is a numeric value

Revision history for this message
Phillip Kelly (pjk-1966) said :
#8

I understand your concept but the value of balance is unicode and I need it to be integer.
I know I should be able to convert by:-

balance = int(balance) #now it is numeric integer

but this doesnt work by itsself.

Unless I need to do something like include the command in a function like :-

in this example http://stackoverflow.com/questions/6393635/valueerror-invalid-literal-for-int-with-base-10

I'm really not sure how to resolve it.

Revision history for this message
RaiMan (raimund-hocke) said :
#9

uups sorry, my fault ;-)

def getValue( text ):
    m = re.match("[^0-9]*([0-9]*)", text) # the value text excluding fraction
    try:
        return int(m.group(1)) # returns the found value as integer
    except:
        print "could not extract number:", text
        return -1 # returns -1 if any problems

Revision history for this message
Phillip Kelly (pjk-1966) said :
#10

Ok, I'm getting somewhere but I have a completely new problem.
An error occurs during the int(m.group(1)) command due to the fact the my re.match command includes "," for every thousand digits.

m = re.match("[^0-9]*([0-9]*\,?[0-9]*)", text)

I am going to try and find a way to remove or ignore the "," so that the text can be assigned as an integer.

Revision history for this message
RaiMan (raimund-hocke) said :
#11

Since you are talking about Integers, you might have noticed, that in my suggestion comment #9 I completely ignored the fraction part of the text.

So if you need a decimal value:

def getValue( text ):
    m = re.match("[^0-9]*([0-9]*[.,]?[0-9]*)", text) # the value text excluding fraction
    try:
        return float(m.group(1)) # returns the found value as integer
    except:
        print "could not extract number:", text
        return -1 # returns -1 if any problems

Revision history for this message
Phillip Kelly (pjk-1966) said :
#12

I did understand that there is no need for the fraction yes. But I think my problem is now that there is a thousand commer included in the string and I need to remove it.

If I understand your revised operators, [.,]? will actually include the commer and decimal point rather than excluding them.

Revision history for this message
RaiMan (raimund-hocke) said :
#13

The version from comment #9 makes an integer value (ignoring any fraction or whatever comes after the first consecutive range of digits)

whereas the version from comment #11 returns a float (decimal value), if the text contains a fraction like .00 or ,00. Otherwise it is a decimal value with fraction 0.

This is the magic of RegularExpressionMatching: you define what you want to see (the whole pattern) and what you want to have extracted (the parts enclosed in braces () numbered 1+ from left to right). It is very powerful, but somewhat mysthic ;-)

Revision history for this message
Phillip Kelly (pjk-1966) said :
#14

Thank you for the detailed discussion.
Your solution in #9 does not acctually yield the whole number, rather it yields only the digits preceeding the comma. All remaining digits are dropped.

In my case the balance read is USD2,750.00 which using #9 results in "2"

Revision history for this message
RaiMan (raimund-hocke) said :
#15

ok, thats new ;-)

we have to tweak the regex somehow

Supposing the max number is 999,999.00 ;-)

def getValue( text ):
    m = re.match("[^0-9]*([0-9]*[,]?[0-9]*[.]?[0-9]*)", text) # the full value
    try:
        return float(m.group(1).replace(",", "")) # returns the found value as decimal
    except:
        print "could not extract number:", text
        return -1 # returns -1 if any problems

Revision history for this message
Phillip Kelly (pjk-1966) said :
#16

Thanks RaiMan, that solved my question.

Revision history for this message
Phillip Kelly (pjk-1966) said :
#17

I have another question on a different issue. Let me know if you think I should just open a new question for it.

I am trying to use a flag to decide where to place by bet in an online casino baccarat game.

the flag should have 2 possible values 0 or 1, 0 for a player bet and one for a banker bet.

the code I am using to toggle the flag does not seem to work :-

def toggle_bet(bet_int, offset):
    mask = 1 << offset
    return(bet_int ^ mask)

the code I am using to initiate the toddle is :-

if exists("win_image.png"):
    toggle_bet(bet_bit, 0)

the code I am using to test the flag is :-

if toggle_bet(0,0):
    do something

what do you think is wrong ?

Revision history for this message
RaiMan (raimund-hocke) said :
#18

Woooow, shuffling bits :--))

for 2-state situations we have the boolean True and False:

betPlayer = True # betPlayer = False means banker bet

if exists("win_image.png"):
    betPlayer = True

if betPlayer:
    do something

Revision history for this message
RaiMan (raimund-hocke) said :
#19

if not betPlayer:
     do something as banker bet

Revision history for this message
Phillip Kelly (pjk-1966) said :
#20

Thanks RaiMan, that solved my question.

Revision history for this message
Phillip Kelly (pjk-1966) said :
#21

RaiMan I am looking for a suggestion from you regarding logic control.
In order to make use of global variables during logic operations it is necessary to perform the logic outside of a function.

At the beginning of my logic I get an input from the user to control the number of hands to be played

hands_to_play = input("hands To Play)
For hands_to_play in xrange(0, 9999):
         #/ then begin a series of logic operations

        if #/some logic :
            #/perform operations
             hands_to_play = hands_to_play - 1

         if hands_to_play <= 0
             exit

Is this a suitable method of program control to include all of the logic operations inside the for loop to enable the use of global variables ?

Revision history for this message
Phillip Kelly (pjk-1966) said :
#22

should this work ?

if exists("win_img.png") and not exists("push_img.png"):
            hands_to_play = hands_to_play - 1
            if betseq == 1:
                   if bet_site == player: #/player and banker variables are screen locales
                       bet_site = banker

                   elif bet_site == banker:
                         bet_site = player

I am using screen locales for player and banker variables to save system resources.

Revision history for this message
Phillip Kelly (pjk-1966) said :
#23

it lives !! the logic works

Revision history for this message
RaiMan (raimund-hocke) said :
#24

--1. forget about system resources ...
... we are working with high potential software systems today and taking care for saving system resources is the job of the programmers of these software systems.
So script whatever you like and however you want.

--2. locals vs globals in Python/Jython
in a script all variables are global and hence can be used all over the place.
they are defined by using them the first time on the left side of a =
Local variables only come into existence inside a def(): the parameter variables are local and each variable that inside the def is on the left side of a = (and because of this a local variable can hide a global variable inside the def)

nameGlobal = 1

def function(nameParameter):
     nameLocal = nameParameter +1 # new local variable, not known outside def
     nameGlobal = nameLocal # new local - does not use global var
     return nameLocal

print function(nameGlobal) # prints 2
print nameGlobal # prints 1
nameGlobal = function(nameGlobal)
print nameGlobal # prints 2

So forget about locals and globals when scripting without using def()'s

When making def()s,
1. all you need inside from outside should be a parameter and
2. changes to the outside world should be done using the return value.

To reduce complexity, it is common with rule 1, that globals are used inside the def on the right side of expressions without having them specified as parameters (but this is always a risk for hidden problems later).

--3. your snippet

if exists("win_img.png", 0) and not exists("push_img.png", 0): # see comment
            hands_to_play = hands_to_play - 1
            if betseq == 1:
                   if bet_site == player: bet_site = banker
                   else: bet_site = player

comment: a search for a visual object in the standard waits 3 seconds before giving up. In your version the if would take in the worst case up to 8 seconds (2 * 3secs + search times) to evaluate
using exists(image, 0) returns after the first trial, so my version in the worst case takes 2 seconds.

Revision history for this message
Phillip Kelly (pjk-1966) said :
#25

Ok I will make your suggested changes.
But I have another problem it seems. I handle the bet placement by passing the bet screen locale to a function.

def bet_a1(place):
    for i in xrange(0, bet_size):
        click(place)
    else:
        click("deal_img.png", 0)

This works fine except that after several hours of play Sikuli exits with the following error :-

[error] TypeError ( an integer is required )

for hands_to_play in xrange(0, 9999999):
    if bet_record.count(300) >= 1:
        exit
    if not exists("win_img.png", 0) and not exists("push_img.png", 0):
        if betseq == 1:
            bet_a1(bet_site) #/ error occurs in this line due to value of bet_site which is a screen locale
            wait(5)
            hands_to_play = hands_to_play - 1

The screen locale are defined

def banker_y(match):
    return match.y
banker = findAll("banker_img.png")
sorted_banker = sorted(banker, key=banker_y)
for icon in sorted_banker:
    exists(banker)
    banker = find("banker_img.png")

def player_y(match):
    return match.y
player = findAll("player_img.png")
sorted_player = sorted(player, key=player_y)
for icon in sorted_player:
    exists(player)
    player = find("player_img.png")

At some point I can not avoid using screen locales to place bets. Even if I do use true and false flags for bet placement decisions I still have to use screen locales to place the bets.
The variable bet_site I use I think is really nice because I can let the logic determine where to place the bet.

Revision history for this message
RaiMan (raimund-hocke) said :
#26

---1. not clear what that should do?

banker = findAll("banker_img.png") # this are matches
sorted_banker = sorted(banker, key=lambda m: m.y) # this are sorted matches
for icon in sorted_banker: now you run through the sorted matches, but do not use the match (icon) in the loop
    exists(banker) # exists not assigned or in an if does not make sense (no effect)
    banker = find("banker_img.png") # what do you expect here? you already searched for ALL banker_img

---2. screen locale
pls. try to explain what this should be

Revision history for this message
Phillip Kelly (pjk-1966) said :
#27

The sorted match code is from the manual listed under the fandAll() statement
http://doc.sikuli.org/region.html#extracting-text-from-a-region

The code overcomes the problem of failed fina() searches, and also allowed a variable to be passed and returned such as banker or player. These banker or player variables return the screen location to place the bet which is wait I mean by screen locale.

I might not have the code exactly right but it works great. I tried changing it several times to things like :-

banker = getLastMatch() #/ didnt work

exists(banker) #/ this is my failsafe to prove the sorted match returned a value

banker = find(banker_img.png) #/ assigns the screen locale to banker and returns it

Revision history for this message
RaiMan (raimund-hocke) said :
#28

If your code does what you expect: fine ;-)
all the best.