Can't match little detail

Asked by Gergely Brautigam

Hi Guys!

I have a simple question. I'm trying to match a header on a web page. To see if it is a match with an already saved picture.

I'm doing this in java.

So i have one picture of the header while the user is not logged in. Then i have a second on which he is logged in.

The exception is that in the upper right corner there is a message : Welcome *username*.

And it says that the two pictures match.

So the question is, what is the minimum amount of change that the algorithm can detect or does not detect. So a welcome message should be triggered as not the same picture not?

Thanks,
Gergely.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Gergely Brautigam
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

Have you played around with the minimum similarity?
Did you restrict the find to a small region around the message?
Did you test it in the IDE using the preview window?

Revision history for this message
Gergely Brautigam (skarlso777) said :
#2

Hi!

Yeah i forgot that i'm using Java as tool. I did try to use the ide but when i'm setting a higher percentage then it doesn't detect it at all. I mean the header.

So in java i'm just trying to wait for the picture and it says that it found it although some minimal changes did happen.

BUT! i tired to take it apart. I checked the header and then i checked the line in which the text is next. So i've got two pictures. One for the header ( two for the show :D ) and second for the line with the welcome message. This way it does work oddly...

I did not restrict the find since i thought i have a picture of the whole header ( because i need to check the whole header for small changes like div miss behaving and things like that. ).

Hope i answered all questions. :)
Gergely.

Revision history for this message
RaiMan (raimund-hocke) said :
#3

IDE:
you are right, the feature to restrict the preview to a smaller region is still missing. so especially with small patterns the feedback in the preview may be misleading, since there is a possibility, that the small pattern is found somewhere on the screen outside of the region you are targeting.

Restricting the find:
With many workflows I found, that telling the find where to search on the screen, makes it more stable and faster. so if you have a chance to somehow know the region, where to search on the screen, you are on the sunnier side of life.

During development of a workflow, the easiest way is to use selectRegion(), to tell the script, where to look:
r = selectRegion("Select the focus area!")
if not r.exists(image-of-login-text, 0):
   popup("Not Found")
   exit()
(to be adapted for java slang)

I always use the timing 0 with exists() in situations, where something should be there, so it only loos once and does not try until the waiting timeout.

Then if you know that your workflow runs with the support of selectRegion(), you can implement something, that finds a characteristic image and calculates the region based on the match (in most cases at least positions of things we are looking for are fixed somehow)

Generally:
Working with Sikuli is totally different from working with a tool like Selenium: you don't have a chance, to find out the inner logic of the web page in terms of <html/css> building blocks.

its the opposite of WYSIWYG: you find eventually what you see --- YFEWYS :-)

Revision history for this message
Gergely Brautigam (skarlso777) said :
#4

Thanks for the answer RaiMan!!

I don't have a chance to know where the header will be when i'm running my test because on every OS or machine it runs on it will be displayed in a different place. So i don't know that one.

But what i eventually did was just using smaller images. Which is not good if i want to check a particularly big area. Say, a big display table or div which should be intact. Then if it differs only a really small amount like going over a table or flowing over to another div or something. Then i wouldn't be able to detect it right?!

Btw i integrated Sikuli into my Selenium tests. So now i have control and visual. :) Nice thing to have. :)

Gergely.

Revision history for this message
Tsung-Hsiang Chang (vgod) said :
#5

Hi Gergely,

I am interested in how you integrated Sikuli into Selenium tests. Can you share more details of that?
It would be great if we can write this down and share to more people. :)

Revision history for this message
Gergely Brautigam (skarlso777) said :
#6

hi Tsung!

It's noting complicated really. :)

I have a framework that uses a combined set of tools. I'm using xml serialization, selenium, testng and reportng to test a web application. It's a good enough framework. It has modularization, ant support, Hudson integration. Everything you need for contentious integration and nightly builds.

This is all written in java. And it is packed correctly into little projects. The base framework that has utilities. Pages mapped as objects and tests and then the other projects based on the versions versions of the web page. The pages are mapped into xml files which are then serialized into objects and used by the framework so if the web page changes it only changes in one place.

Then comes Sikuli. I created some testng tests called ui tests. I used the java example located in the manual.

Then i used selenium to navigate to the page i want to test and used sikuli to verify that the css and look and feel of the page is good enough. Since i'm trying to check little things it doesn't work so i'm using pictures of little parts of the web page and i'm comparing small pieces after one another.

So you see it is nothing very complicated or java webrobot like or stuff like that i'm just using selenium to get where i want and then i'm using sikuli to check what i need to see. :)

Was that a good enough answer? :)

Revision history for this message
Gergely Brautigam (skarlso777) said :
#7

Oh and users are created with an object factory pattern. And there are some hider factories to hide and encapsulate selenium. So selenium is only responsible for himself. Which is how the whole framework is created. Everybody is hidden from everybody as best as it can be.

Revision history for this message
Yanan (baoji58) said :
#8

Hi Gergely,

It would be great if you can provide more detail and sample code.

Thanks,
Yanan

On Mon, Jun 21, 2010 at 3:55 PM, Gergely Brautigam <
<email address hidden>> wrote:

> Question #115002 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/115002
>
> Gergely Brautigam gave more information on the question:
> Oh and users are created with an object factory pattern. And there are
> some hider factories to hide and encapsulate selenium. So selenium is
> only responsible for himself. Which is how the whole framework is
> created. Everybody is hidden from everybody as best as it can be.
>
> --
> You received this question notification because you are an answer
> contact for Sikuli.
>

Revision history for this message
Gergely Brautigam (skarlso777) said :
#9

Hi Yanan!

I was thinking of actually opensourcing the frameworks skeleton with one test example.

But for that to happen i need to finish the manual which is quite big. :)

So please give me some time and will link to the project when i'm done with it.

Is that okey so? Or do you want a smaller version of it ( there isn't really... :)))) )

Gergely.

Revision history for this message
Yanan (baoji58) said :
#10

Great idea*, *Gergely!* *
Can't wait to see it*!
*
Yanan*
*
On Tue, Jun 22, 2010 at 3:06 PM, Gergely Brautigam <
<email address hidden>> wrote:

> Question #115002 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/115002
>
> Status: Answered => Solved
>
> Gergely Brautigam confirmed that the question is solved:
> Hi Yanan!
>
> I was thinking of actually opensourcing the frameworks skeleton with one
> test example.
>
> But for that to happen i need to finish the manual which is quite big.
> :)
>
> So please give me some time and will link to the project when i'm done
> with it.
>
> Is that okey so? Or do you want a smaller version of it ( there isn't
> really... :)))) )
>
> Gergely.
>
> --
> You received this question notification because you are an answer
> contact for Sikuli.
>