need some basic hints to get on the road with Sikuli

Asked by Benjii Tiu

Hi there.... how do you do combination of keypad on sikuli. You see i wanted to do ctrl + w (cose current tab) instead of clicking x image since there are 2 x image in browser.

Also, how to do if there are many same like images on the page? What should i do in order to pinpoint where i would want to click on the page?

Can i do it also with just colors of the letter? say like this, there are a bunch of letters colored blue in this particular page that i wanted to click since there is no image of it, only color blue.

Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
Benjii Tiu
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

Generally: there are docs, where some of your questions are answered.

--- combination of keypad
http://sikuli.org/docx/keys.html#key-constants and
http://sikuli.org/docx/region.html#Region.type

--- if there are many same like images on the page
http://sikuli.org/docx/region.html#Region.findAll

--- Can i do it also with just colors of the letter?
Not really clear what you mean.
you can only click on images. so if it is possible to make an image of the single blue letter, you might try to search it and click it.

Revision history for this message
Benjii Tiu (jn-tongko) said :
#2

Saw that combination of keypad before. I tried it using this one
type(Key.CTRL + W)
and it did not work... how should i do the combination thing? the key will work if its not combination, like type(Key.TAB) or type(Key.HOME)
I do hope you could give me just the answer to this, i guess im doing something wrong here, cause its not working

Regarding the blue letter, this is the screen for it.
https://www.dropbox.com/s/n49gu3kgnlhh3yu/letter%20in%20blue.png
I put a line inside which i want to click, all blue are link text. that's why they are blue. 1 page consist of 30 rows i think, as you can see, there is also blue on the left and right side. What i want to do is to click them maybe randomly from that region, or do them starting from the top going to the bottom. Cause once you click a particular blue words, it will become gray. Could sikuli do that?

Would probably be good if you gave me a sample on how to do a region.
Like this sample.
https://www.dropbox.com/s/rhzimlaj67l2h7x/same%20images.png
I wanted just 12 same green icons, i just wanted sikuli to click those 3 top rows always. can i do that with sikuli? How do you that? I saw a region icon on sikuli, they are on the same row with capture, so how do i do that?
region(image)
click(image)
is that how you do it with region?

I am learning so much faster by getting my own problems fixed and i just want to automate things so i use sikuli. For the first question i did here about looping, like you, the expert also gave me a link https://answers.launchpad.net/sikuli/+faq/1437 but i did not know how to do it until he did put the sample on how to do it, base on the problem i had. That was then that i know how to do loop. Here is the link on it https://answers.launchpad.net/sikuli/+question/204946

I guess what i'm trying to say is that i learn through hands on base on problems i encounter on how to do it with sikuli. Its so much better if there are pictures on the samples you gave and explain or give the code in sikuli.

Revision history for this message
RaiMan (raimund-hocke) said :
#3

--- type
The examples in the doc tell you:
type("w", KeyModifier.CTRL) # lowercase w !

--- blue letters problem
this is not possible to solve with Sikuli using image search, since I guess, that the text in the marked column changes.
This a typical "click through a table column" problem, that can normally only be solved by clicking an calculated points on the screen.
You somehow have to find out the uppermost click position (first line) and the click the next ones using an offset to the fist one.
If you tell me, how I can get this live on my screen, I can give you more tips and samples.

--- green buttons
this could be solved the same way, since it is also a regular geometric GUI pattern.
But in this case you could use findAll().
And it is exactly the approach as mentioned at
http://sikuli.org/docx/region.html#Region.findAll

buttons = findAll("imageOfGreenButton.png")
sorted_buttons = sorted(buttons, key=lambda m: m.y*10000+m.x)
for button in sorted_buttons:
    button.highlight(1)

This will (hopefully ;-) find all green buttons and click them in the order from left-to-right/top-to-bottom

or if you only want to click the first 3:
for i in range(3):
    sorted_buttons[i].highlight(1)

--- is that how you do it with region?
not really.
A region is an area with a fixed position and size on the screen.
You can only use it to click on it, if you are sure, that the region is always at the same position:

reg = Region(x,y,w,h) # you might have defined it with the above mentioned RegionButton
click(reg) # would click the center of the region

Regions are usually used, to restrict the search area for speed and accuracy.

example with your first shot:

top = "FeaturedListing.png"
mTop = find(top)
mTop.highlight(1)
bottom = "FollowUs.png"
mBottom = find(bottom)
mBottom.highlight(1)
area = Region(mTop.x, mTop.y+mTop.h, mTop.w, mBottom.y-mTop.y-mTop.h)
area = area.nearby(10) # to add some margin
area.highlight(1)
# combined with the findAll() example
buttons = area.findAll("Button10.png")
sorted_buttons = sorted(buttons, key=lambda m: m.y*10000+m.x)
for button in sorted_buttons:
    button.highlight(1)

Hope it helps;-)

Revision history for this message
Benjii Tiu (jn-tongko) said :
#4

My mind is scrambling... I should have just asked questions 1 by 1... since i do also work online, hahahhaa. I will try to solve my problems base on the samples you gave which is good for me. Regarding that blue letter, i can't just give you the logins d]etails of my account on it right, since this is public. If you can send me an email, i'll give you the details on how to login Sir. Thanks a lot for all the help... Is this voluntary help you are doing at this community?

Revision history for this message
RaiMan (raimund-hocke) said :
#5

So if it is not possible, to look at the sample, without logging in somewhere, then just send me a screenshot of the whole window or even screen to my mail at https://launchpad.net/~raimund-hocke