SikuliX

Return the dimensions of an image that's just been found

Asked by arminius

So this is an example of the code I'm working with, the real code has twenty images, but I've simplified it to three.

mySimilarity = 0.86
count = 0
main_address = r"C:\Users\USER_ADDRESS\4thloop"
while True:# Searching......at the 4th loop. LAST ADDED 13 04 2018
  while (count < 10): # Reduce the Match by 9 percent.
 if exists(Pattern(Pattern("1519473664449.png").targetOffset(1,-22)).similar(mySimilarity)):
   print('found 1st image at similarity %f') % mySimilarity
   first_image_used = main_address + r"\first_image_used.txt"
   file = open(first_image_used)
   current_amount_of_times_image_used = file.read()
   file.close()
   new_current_amount_of_times_image_used = int(1) + int(current_amount_of_times_image_used)
   print('Number of times this image has been found %f') % new_current_amount_of_times_image_used
   text = str(new_current_amount_of_times_image_used)
   f = open(first_image_used, 'w')
   f.write(text)
   f.flush()
   f.close()
   break
 elif exists(Pattern(Pattern("1525270211713.png").targetOffset(1,-21)).similar(mySimilarity)):
   print('found 2nd image at similarity %f') % mySimilarity
   second_image_used = main_address + r"\second_image_used.txt"
   file = open(second_image_used)
   current_amount_of_times_image_used = file.read()
   file.close()
   new_current_amount_of_times_image_used = int(1) + int(current_amount_of_times_image_used)
   print('Number of times this image has been found %f') % new_current_amount_of_times_image_used
   text = str(new_current_amount_of_times_image_used)
   f = open(second_image_used, 'w')
   f.write(text)
   f.flush()
   f.close()
   break
 elif exists(Pattern(Pattern("1525274687366.png").targetOffset(1,-22)).similar(mySimilarity)):
   print('found 3rd image at similarity %f') % mySimilarity
   third_image_used = main_address + r"\third_image_used.txt"
   file = open(third_image_used)
   current_amount_of_times_image_used = file.read()
   file.close()
   new_current_amount_of_times_image_used = int(1) + int(current_amount_of_times_image_used)
   print('Number of times this image has been found %f') % new_current_amount_of_times_image_used
   text = str(new_current_amount_of_times_image_used)
   f = open(third_image_used, 'w')
   f.write(text)
   f.flush()
   f.close()
   break
 else:
   count = count + 1
   mySimilarity = mySimilarity - 0.01 # reduce by 1%
   print('mySimilarity is %f') % mySimilarity
   if count > 8:
  print('none of the images found.')
  popup('none of the images found.')
   continue # retry from the beginning
  break

When "none of the images found" is printed excessively I go looking for the corresponding text file with a zero in it, I replace the image, then start the program over from the beginning, all up that wastes about 10 minutes.
So I've discovered the screen capture feature, so I figure I can automate the process by having a new screenshot taken of images as they are found, but I'm not sure how to return the dimensions of a found image.
Once I know how to return the dimensions of a found image I can update the least recognized images.
Hope that made sense.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
masuo
Solved:
Last query:
Last reply:
Revision history for this message
Best masuo (masuo-ohara) said : 		#1

When an image is found you can get dimensions of an image.

m = exists("image01.png")
if m:
    print m
    print m.getX(), m.getY(), m.getW(), m.getH()

But I can't understand you want to capture a new image when an image is found.

BTW:
SikuliX IDE have pattern setting feature.
Do you know this feature?
Click an image in editer pane of SikuliX IDE, pattern setting window open.
There are matching preview tab and target offset tab in pattern setting window.
You can adjust similarity manually by using matching preview feature of pattern setting window.

By using matching preview feature and target offset feature, pattern object is created like this.
exists(Pattern("image01.png").similar(0.80).targetOffset(10,0))

Revision history for this message
masuo (masuo-ohara) said : 		#2

Additional Information:
http://sikulix-2014.readthedocs.io/en/latest/match.html?#Match
Since class Match extends class Region, all methods of class Region can be used with a match object.

http://sikulix-2014.readthedocs.io/en/latest/region.html#Region.getX

Revision history for this message
arminius (arminius75) said : 		#3

"But I can't understand you want to capture a new image when an image is found."

The images I'm searching for are dynamic, the locations on the screen can change, the colors can change.
I can't lower the match percent to much or it will click on the wrong thing.
The current system I have that works well is to start with 3 out of 20 images to start, starting at high match percentage, and slowly becoming less accurate by 1 percentage point as the loop cycles around.
After a few days the original 3 become less accurate, I don't want to replace them because in time they will return to being more accurate, so I add another 3.
The current process is just very time intensive, if I can program it to automatically take a new screenshot with the last image to be found's "m.getX(), m.getY(), m.getW(), m.getH()" and then send that picture to replace an image that has never been seen before.
Then slowly I will have a library of images built up that I didn't have to do myself.

Hope that made sense.

It probably didn't.

Imagine the first image in the loop is of an apple, and the second is of a banana, and the 3rd is an orange.
You want it to only find images of apples.
So the first image is linked to a text file that has the number 34, because it's been seen 34 other times.
The other two have zero in them.
When we see the first image at match percentage of 54, we take a brand new more accurate snapshot of the apple, we then see which image is at zero, and replace the 3rd image of an orange with the latest image of the apple.
So far I've been doing all that manually, so I'm wanting to code so that it does it as part of the program.
Eventually it should all be apples.

Revision history for this message
arminius (arminius75) said : 		#4

This is my current working test.

    import shutil
    import os
    screenshotsDir = "C:\Users\ADDRESS_PATH\1stloop"
    wait(1)
    mySimilarity = 0.90
    Reg = Region(413,242,519,366)
    search = True
    while search:
      if Reg.exists(Pattern(Pattern("1525655012965.png").targetOffset(-1,-19)).similar(mySimilarity)):
        m = (Reg.getLastMatch())
        Xis = m.getX()
        Yis = m.getY()
        Wis = m.getW()
        His = m.getH()
        print m.getX(), m.getY(), m.getW(), m.getH()
        print(Xis)
        print(Yis)
        print(Wis)
        print(His)
        img = capture(Xis, Yis, Wis, His)
        shutil.move(img, os.path.join(screenshotsDir, "\First_image_used.png"))
        hover(Reg.getLastMatch())
        search = False
    wait(5)
    print('Finished')

The program executes without any bugs, goes right to the end, but the First_image_used.png is not being changed at all.

Revision history for this message
arminius (arminius75) said : 		#5

Thanks masuo, that solved my question.