SikuliX

How to differentiate 2 similar images based on color???

Asked by Vaishali

I m using sikuli script in eclipse.. in that is tr any method to differentiate 2 similar images based on color...

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Eugene S
Solved:
Last query:
Last reply:

This question was reopened

Revision history for this message
Test (c4456517) said : 		#1

Why don't you just use the 2 different images?

Mind showing us how the images look like?

Revision history for this message
Vaishali (vaishoo-vasu) said : 		#2

I want to click on an image with perticular color.. bt tr is another similar image bt in diffrent colors..

Revision history for this message
Test (c4456517) said : 		#3

I think you should elaborate on your situation further.

If you want to click on an image with a particular colour, why can't you place that image within click() like click(image.png)? If there's another image like that but in a different colour, I highly doubt Sikuli will detect that on your screen since it's a different colour.

That being said, you might want to alter the similarity as such: click(image.png).similar(0.8) where the number within the parentheses ranges from 0 - 1 (1 being of maximum similarity).

More here: http://doc.sikuli.org/pattern.html?highlight=similar#Pattern.similar

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

lets say you have 2 buttons (btn1 and btn2) then the following workflow will lead to the right Pattern setup:

#take the 2 btn shots:
imgBtn1 = "<shot of btn1>
imgBtn2 = "<shot of btn2>
print "btn1 in around btn1", selectRegion().find(imgBtn1) # select a region around btn1
print "btn1 in around btn2", selectRegion().find(imgBtn1) # select a region around btn2
print "btn2 in around btn1", selectRegion().find(imgBtn2) # select a region around btn1
print "btn2 in around btn2", selectRegion().find(imgBtn2) # select a region around btn2

take care, that the selected regions are larger than the buttons on all sides.

This will give you the similarity scores of all 4 possible situations.
Now you can decide about the threshold similarity to use with the pattern.

click(Pattern(imgBtn1).similar(0.95)) # 0.95 is an example

Generally it is always a possibility with version 1.0.1+ (RC3 has a bug here) to try with
click(Pattern(imgBtn1).exact())
but this approach needs "optimal" image shots: as little background as possible, concentrate on the key visual aspects of the image (for buttons this usually is a shot, that contains not more than the button's frame).

And even more general: Use the IDE's Preview, to find out the similarity threshold.

Revision history for this message
Vaishali (vaishoo-vasu) said : 		#5

Actually Both are similar images and tr is only a slight difference in color.. for example next button in gmail... when it reaches the last email it should stop clicking the next button..

Revision history for this message
RaiMan (raimund-hocke) said : 		#6

what about the test and results with comment #4?

Revision history for this message
Vaishali (vaishoo-vasu) said : 		#7

ya i tried #4, as the images are around the same region its difficult to identify it... According to my question a single button changes to disable state after reaching the end.. so i need to stop the process when the button is disabled..

Revision history for this message
RaiMan (raimund-hocke) said : 		#9

... ya i tried #4
and what was the outcome?

could you just paste the results? would help to help you ;-)

Revision history for this message
Best Eugene S (shragovich) said : 		#10

When trying to find a solution to similar problem I have noticed that often there are slight differences in similarity coefficient in 2 similar images of different color.

So what you can do is that:
1. Create a pattern of one of the images. For example "image1.png"
2. Locate it on the screen (within the relevant region), for example:
   find("something.png").nearby(x).find("image1.png)

You will get something like that:
   M[529,157 123x10]@S(0)[0,0 1280x1024] S:1.00 Center:590,162 for 1.0 secs

3. Take a note of whatever value appears in the Similarity (S) field.
4. Now use the same pattern to find the other image (for esxample "image2.png").
5. Again, take a look at whatever value is appearing in the Similarity (S) field.

At this stage, hopefully, you will see that the similarity value is slightly different that 1.00. For example:
   M[512,134 123x10]@S(0)[0,0 1280x1024] S:0.97 Center:570,142 for 1.0 secs

Now, you can exploit the fact that there is a difference between the similarity values of the 2 images and base your actions on that.

To check the similarity coefficient of a certain pattern you can use this command:

   im = find("image1.png")
   similarityCoeff = im.getScore()

Regards,
Eugene

Revision history for this message
Vaishali (vaishoo-vasu) said : 		#11

When i tried it.. #4 When it reached the end of the file and the button was disabled bt.. it didn't differentiate between the 2 buttons.. and it continuously stared clicking the same button.. so i need to stop the program using task manager.. and it didn't through any error...

Revision history for this message
RaiMan (raimund-hocke) said : 		#12

if you really tried the workflow of comment #4 or a similar approach like in comment #10, then you have some information about similarity scores in the different situations.

This is what we want to see here to help you further.

BTW: these tests should be done in a new script. do not mix up this with your planned workflow.

Revision history for this message
Vaishali (vaishoo-vasu) said : 		#13

Thanks Eugene S, that solved my question.