SikuliX

find image and type pattern

Asked by ben

iu am using sikuli version 1.1.3
I want to find image pattern and type out accordingly
only 6 pattern: up arrow, down narrow, left arrow, right arrow, letter j and letter k

the image pattern could be up,up,k the code will detect and type up,up,k
the image pattern could be down,left j the code will detect and type out down,down,j

my code is to find the image and sort it using the x coordinate, then type out

patterns = [Pattern("up.png").similar(0.85),Pattern("down.png").similar(0.85),Pattern("left.png").similar(0.85),Pattern("right.png").similar(0.85),Pattern("j.png").similar(0.85),Pattern("k.png").similar(0.85)]
start = Pattern("start.png").similar(0.90)
mm = []
all_icons = []

def by_x(match):
   return match.x

def by_sort(icons):
    # sort the icons by their x coordinates and put them into a new variable sorted_icons
    sorted_icons = sorted(icons, key=by_x)
    for icon in sorted_icons:
        popup(icon.getIndex())

def recycleAll(x,name):
# --- using exception handling
# every not found in the try block will switch to the except block
    try:
        icons = findAll(x)
        while icons.hasNext(): # loop as long there is a first and more matches
            all_icons.append(icons.next())
        all_icons.append(name)
        icons.destroy()
    except FindFailed:
        pass # we miss it

while True:#This is the main loop of the application.
    if exists(start):
        for x in patterns:
            recycleAll(x,x.getFilename())
        by_sort(all_icons);
        #by_key(mm)
        mm = []
        all_icons = []
        pass
    else:
        pass

Having problem after sorting them by x coordinate, not sure how to detect and type

Question information

Language:
English Edit question
Status:
Expired
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

this would be my solution (not tested ;-)

patterns = [Pattern("up.png").similar(0.85),Pattern("down.png").similar(0.85),Pattern("left.png").similar(0.85),Pattern("right.png").similar(0.85),Pattern("j.png").similar(0.85),Pattern("k.png").similar(0.85)]
start = Pattern("start.png").similar(0.90)
mm = []
iconsFound = []

def findAllIcons(pattern):
   try:
       icons = findAll(pattern)
       while icons.hasNext(): # loop as long there is a first and more matches
           iconsFound.append(icons.next())
       icons.destroy()
   except FindFailed:
       pass # we miss it

while exists(start):#This is the main loop of the application.
   for pattern in patterns:
       findAllIcons(pattern)
   iconsFound.sort(key = lambda match: match.x)
   message = ""
   for match in iconsFound:
       message += os.path.basename(match.getImageFilename()).replace(".png", "")
   print message
   mm = []
   all_icons = []

Revision history for this message
ben (ben862000) said : 		#2

starttime = time.time()
patterns = [Pattern("up.png").similar(0.85),Pattern("down.png").similar(0.85),Pattern("left.png").similar(0.85),Pattern("right.png").similar(0.85),Pattern("j.png").similar(0.85),Pattern("k.png").similar(0.85)]
start = Pattern("start.png").similar(0.90)
aRegion = Region(600,600,800,300)
iconsFound = []

def findAllIcons(pattern):
    try:
        icons = aRegion.findAll(pattern)
        while icons.hasNext(): # loop as long there is a first and more matches
            iconsFound.append(icons.next())
        icons.destroy()
    except FindFailed:
        pass # we miss it

def typePattern():
    iconsFound.sort(key = lambda match: match.x)
    for p in iconsFound:
        check = os.path.basename(p.getImageFilename()).replace(".png", "")
        if(check == "up"):
            type(Key.UP)
        if(check == "down"):
            type(Key.DOWN)
        if(check == "left"):
            type(Key.LEFT)
        if(check == "right"):
            type(Key.RIGHT)
        if(check == "j"):
            type('j')
        if(check == "k"):
            type('k')

while True:
    if exists(start):#This is the main loop of the application.
        for pattern in patterns:
            findAllIcons(pattern)
        typePattern()
        iconsFound = []
        print "Elapsed: %d seconds"%(int(time.time()-starttime))
        wait(1)

Thanks for your help.
I have complete my code, but i am finding way to optimize it

it take about 27 sec to complete for a single pattern for example down arrow.
can it shorten to less than 4 sec?

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

can there be more than one of the 6 possible patterns be visible at the same time?

the search area (aRegion) is rather large?

Revision history for this message
ben (ben862000) said : 		#4

yes, there can be 1 to 7 possible pattern

up or up,up,down,down,up,up,down

Revision history for this message
ben (ben862000) said : 		#5

i can decrease aRegion = Region(730,740,470,70)

Revision history for this message
Launchpad Janitor (janitor) said : 		#6

This question was expired because it remained in the 'Open' state without activity for the last 15 days.