how to change default screen to a custom region?

Asked by Kopanda

Hi!
Im making a bot for a game on bluestacks and i need it to only look for matches inside bluestacks window, but sometimes it clicks on my desktop or even on a stream that im watching during the farm.
So i tried to solve it like this:
SCREEN = "custom region created via IDE toolbar"

didnt work (i moved bluestacks window outside the defined region and bot still found matches there)

then i tried:
r = "custom region created via IDE toolbar"

and wrote "r." before each exists, wait and click function, it stopped working completely.

Im only using one monitor, also each click, wait and exists functions contain only 1 argument that has an image created via "insert screenshot" button on the IDE toolbar.

Please HALP :O

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Kopanda (kopanda) said :
#1

sooo looks like sikuli is dead

Revision history for this message
Eugene S (shragovich) said :
#2

A region can be defined as simple as:

customReg = find("pattern.png")

then if you want to find something within that region you can act in a similar manner:

wantToFind= customReg.find("anotherPattern.png")

Revision history for this message
Kopanda (kopanda) said :
#3

umm i dont need to define a region through pattern, i need it to be exactly where i say. I believe "Region" tool does that, so i go r = "region made with tool"

then if i go r.exists("pattern.jpg") it will still be looking everywhere on the screen.

Maybe regions dont work with exists command, or im doing something else wrong?

Revision history for this message
Eugene S (shragovich) said :
#4

If you want to set a Region to a constant screen location just create a new Region object while passing the pixel parameters to the constructor. For example:

region = Region(x,y,w,h)

where:
x – x position of a rectangle.
y – y position of a rectangle.
w – width of a rectangle.
h – height of a rectangle.

like described here: http://doc.sikuli.org/region.html#Region.Region

Revision history for this message
Kopanda (kopanda) said :
#5

Thanks, but thats not what i need, im not asking how to define a region - im asking how to make Sikuli work as i described. Am i using the tool wrong?
here is an example: http://imgur.com/2XQ5WEw
 Will it work? If not - why? Thanks for helping me Eugene! :D

Revision history for this message
Eugene S (shragovich) said :
#6

I guess you're right then and I probably do not understand what is the problem you are facing. Would you mind try and explain it again?

As far as I understood your problem you have a specific physical and constant region on your screen that you want to find something within this region, is that correct?

Revision history for this message
Kopanda (kopanda) said :
#7

Yes i want Sikuli to only search for matches within that region to: a) speed up the search and b) avoid accidential matches (while im watching the stream for example). So i used a tool to create region (as shown on screenshot) and then i have a bunch of "If exists()" lines, followed by "click(getLastMatch())" lines and when i tried to define a region (as on the screenshot) and then add "r." in front of each "exists" function - Sikuli was still finding matches and clicking outside the defined region.

Revision history for this message
Eugene S (shragovich) said :
#8

Look, it is very hard to see the graphics on your screenshot but it does look like something that I explained in my initial post. I will go through the steps again.

1. Define a region you want to search and store it in a variable.
2. Create patterns of what you want to find in this region (you can only work with pre-defined patterns in Sikuli)
3. Search or wait for these patterns to appear in the region defined in step 1.

This is the same answer I gave in my first post. Does that make sense? It it doesn't, what part of it is that you have a problem with?

Revision history for this message
Kopanda (kopanda) said :
#9

I have a working program that is searching and finding patterns on the whole screen, it does it's job great, BUT sometimes, very rarely, it finds matches outside the game window, so i want to restrict the search to region.

To do that i first define a region by typing:
r =

after then i press "region button" on the toolbar, program asks me to select a region. I select it, program pastes a mini picture of that region after equals sign in the expression, that means i defined a region and stored it into "r" variable, is that correct? If thats not how the tool works - please correct me.

So if i did everything correct - i just defined a region.

Next i take my almost perfectly working code that is ONLY using "exists" functions to find patterns on the screen, (no other function is looking for patterns on the screen in my code), so then i add "r." in front of every single "exists" function in my code.

If i understand correctly - now Sikuli should look for those patterns not on the entire screen, but only in region that i stored in "r" variable. If that is correct - after that my program should stop finding patterns and clicking outside the region that i stored in "r" variable, but it doesnt - it still finds matches outside the region that i stored in "r" variable and i dont understand why.

Either "exists" function ignores region that it was called by and always searches the entire screen, OR the IDE tool that i marked with an arrow on my screenshot works not as i expect it to work.

In any case i could try and define region differently (for example like you said, but using this method will be problemmatic, since i might want to move the game window and i will need to get coordinates and type them in code again every time, while using UI tool is super easy) and i could replace all "exists" functions with "click" functions and ignore all the "pattern not found" errors, but i really want to understand why my code doesnt work, thats almost as important for me to understand what im doing wrong as to fix the code.

Revision history for this message
Eugene S (shragovich) said :
#10

Well, I think I know what your problem is. When you define a region in Sikuli, it remains constant. Meaning that if you moved it elsewhere, you will have to redefine it (or to find it again) to use. Moreover, you must remember that the ONLY thing that Sikuli does is finds a predefined patterns/regions on the screen precisely as they were saved. So if you captured your region WITH some visual areas around it, then Sikuli sees all that as a single pattern and will look for it exactly. So you must make sure that no unrelated pixels are present in your capture. In some cases, windows can be transparent and unrelated stuff might be visible through the window. this is a problem too.

Please also note that I am not using IDE in my examples, I use Sikuli as script from command line. Hence I do not capture the region like you do from IDE. I do that with print screen and then just store it with my script and refer to it using it's name, for example "pattern.png". This is easier when you want to cut the image precisely in paint form example.

To summarize:
1. If you created a region and it slightly changed - Sikuli won't find it
2. If you created a region and it moved- Sikuli won't find it

If you say that your pattern matched outside the region, that means that you didn't define the region correctly or doing something else funny. Try using find() inside the region rather than exists(). Like: region.find("pattern.png")

Revision history for this message
Kopanda (kopanda) said :
#11

Thanks for all your help Eugene, sadly, i knew everything you said before, and as i said - i know several possible workarounds, but i want to understand what im doing wrong in my particular case (with "exists" function and IDE tool).

Revision history for this message
masuo (masuo-ohara) said :
#12

When click() does not work as I think, I use mouseMove() and click().
But this is in my case. This might not work in your case.

http://sikulix-2014.readthedocs.org/en/latest/scripting.html?highlight=mousemove

[excerpt of the document:]
If the internal timing of the compound mouse functions like click() or dragDrop() is not suitable in your special situation, you might as well build your own functions using the basic mouse functions Region.mouseDown(), Region.mouseMove() and Region.mouseUp()

[example:]
if exists("image") :
   mouseMove("image")
   click("image")

Revision history for this message
Kopanda (kopanda) said :
#13

everything works, thats not the case

Revision history for this message
masuo (masuo-ohara) said :
#14

Your operation to specify the region is correct .
I think your operation to specify the image is wrong.
You are using jpg file for argument.
You should change jpg file to png file.
To do so, use "Take screenshot" button instead of "Insert image" button.
When you use png file , you can set similarity for matching.
Have you been using the "Matching Preview" window?
You can set similarity for matching on the "Matching Preview" window.

Revision history for this message
Kopanda (kopanda) said :
#15

Aw, totally my bad, that was a typo in my message - i am actually using PNG not JPG and as i said my code works, but it works not as i expect it to work - it finds matches outside the region, even tho it seems to me that im doing everything right, therefore it shouldnt.

Revision history for this message
Karl (k-d) said :
#16

You might use:
r = App.open(bluestacks_app).window() #http://sikulix-2014.readthedocs.org/en/latest/appclass.html?highlight=window#window
if r.exists("image1.png")
    r.click("image1.png")

The IDE doesn't seem to update the region definition unless I change something in a script.
The scope of the region doesn't seem to change with indentation, but you can use the following to see what Sikuli thinks the region is:
print r
or
print r.x, r.y, r.w, r.h

Lastly, your screenshot in comment #5 indicates you're using exists incorrectly. Exists does not take a region as a parameter; it only takes a Pattern or string (http://sikulix-2014.readthedocs.org/en/latest/region.html?highlight=click#Region.exists), unlike all of the Region actions, including click, (http://sikulix-2014.readthedocs.org/en/latest/region.html?highlight=click#acting-on-a-region) which take PSMRL.

Revision history for this message
Kopanda (kopanda) said :
#17

Thanks Karl, i will try this.
In comment 5 i just selected region icon as a pattern, i realised that was a bad idea and may confuse some1 after i posted it, but i was lazy enough to take the risk XD

Can you help with this problem?

Provide an answer of your own, or ask Kopanda for more information if necessary.

To post a message you must log in.