SikuliX

i want sikuli to select options closest to the mouse pointer location when there are more then one matches present

Asked by geo

I ran into a major problem while making a script. At present sikuli randomly selects the pictures , i wanted sikuli to select the recognized picture closest to the mouse since there are more than 5 matches , at least on the screen.but since i am new to this i am hopelessly lost ! , how do i get sikuli to select the the picture closest to the mouse location when there are multiple matches. thanks :)
is it possible to achieve this by setting a region at the mouse location every time ?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
geo (gtyson4) said : 		#1

my problem is quite similar to this post "https://answers.launchpad.net/sikuli/+question/225412", i am already using hover()..but i have no idea how to set the program to calculate the nearest distance from the mouse pointer location.

Revision history for this message
Eugene S (shragovich) said : 		#2

Hi,

There are a couple of options I can think about in regards to how to find specific object from multiple similar matches however it depends on how exactly these objects are distributed on the screen and if there are any "anchors" that can be used.

Answering your question directly, there is that option:

1. Use loc = Env.getMouseLocation() to get the mouse location.

2. Create region based on that location:
    reg = Region(loc.getX(), loc.getY(), 1, 1)

3. Try to extend the region, as described here:
     http://doc.sikuli.org/region.html#extending-a-region
     until you cover the area you are interested in.

4. Use highlight(seconds) method to better understand where your region is exactly.

For example:

loc = Env.getMouseLocation()
reg = Region(loc.getX(), loc.getY(), 1, 1).nearby(20).highlight(1)

This will give you a frame around your mouse that might cover one of your patterns.

Please try that and update if you still facing issues.

Cheers,
Eugene

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#3

@Eugene
this might be an heuristic approach, to be able to find a region, that contains the nearest element.

This is a more general solution:

import math

# used by sorted, to calculate the distance to refLocation
# the sort function uses this value for sorting the list elements
def by_nearest(match):
  x = math.fabs(match.getCenter().x - refLocation.x)
  y = math.fabs(match.getCenter().y - refLocation.y)
  distance = int(math.sqrt(x * x + y * y))
  return distance

img = "some-image.png" # the image to look for

refLocation = Env.getMouseLocation() # the reference point

# use findAll(), to find all matches on screen
# and create the sorted list
sortedMatches = sorted(findAll(img), key=by_nearest)

# list contains the elements sorted by distance in ascending order
# hence the first element is the one we are looking for
sortedMatches[0].highlight(2)

Revision history for this message
geo (gtyson4) said : 		#4

Thanks RaiMan, that solved my question.