Screen(1) doesn't seem to work.

Before trying such complex things like observe (btw: not working reliably in RC3 and 1.0.1), you should do some basics.

1. known class names are red (sorry for this historical thing ;-) so scr = Screen(0) is ok, but not needed, since any undotted method addresses Screen(0).

2. selectRegion() as such is not bound to any screen (like capture also. So (see 1.) just using
r = selectRegion()
is always enough.
Only in the moment when you start the select/capture, the selection is restricted to the screen where you started the action.

Do this:
Screen(0).highlight(3)
Screen(1).highlight(3)

This will show you the respective screen with a red frame and print the dimensions.

Ah ha. So what happens when I do that is it highlights the same monitor for both Screen(0) and Screen(1). No wonder I was having trouble with that. I also tried something like c=capture(-400,500,200,200) to grab something on the left monitor (my laptop) and it told me it was out in an area that doesn't exist (or something to that effect).

Do you think I have some sort of configuration thing that Sikuli is not seeing the location of that other screen like it should?

Thanks for the quick response by the way.

By the way, when you say "print the dimensions", where will it print it? Does that assume I'm running from command line or something? I'm running the IDE in this case. I do see it highlight the whole screen (both times in this case). Also I wanted to mention that the observe functions within the changed method are working fine for the region that I select. So all that is fine. I just need to figure out how to work on something over in the other monitor.

Thanks,
Greg

What version of Sikuli?

Sikuli IDE 1.0.1

the following may be more than you need, but thought it might help:

[debug (2/20/14 4:26:56 PM)] RunSetup: SikuliX Setup Build: 1.0.1 12NOV2013120000
[debug (2/20/14 4:26:56 PM)] RunSetup: ... starting with no args given
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Users\ghowell\Documents\SikuliX\libs
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files (x86)\Intel\iCLS Client\
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files\Intel\iCLS Client\
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Windows\system32
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Windows
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Windows\System32\Wbem
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Windows\System32\WindowsPowerShell\v1.0\
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files\Intel\Intel(R) Management Engine Components\DAL
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files\Intel\Intel(R) Management Engine Components\IPT
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: c:\Program Files (x86)\Microsoft SQL Server\90\Tools\binn\
[debug (2/20/14 4:26:56 PM)] RunSetup: syspath: C:\Program Files (x86)\QuickTime\QTSystem\
[debug (2/20/14 4:27:04 PM)] RunSetup: Option UPDATE selected
[debug (2/20/14 4:27:04 PM)] RunSetup: checking for newer versions
[debug (2/20/14 4:27:07 PM)] No version info available at https://dl.dropbox.com/u/42895525/SikuliX
[debug (2/20/14 4:27:09 PM)] RunSetup: You already have the latest version!
[debug (2/20/14 4:27:09 PM)] RunSetup: Completed: checking for newer versions
[debug (2/20/14 4:27:09 PM)] RunSetup: user home: C:\Users\ghowell
[debug (2/20/14 4:27:09 PM)] RunSetup: RunningSystem: Windows 6.1
[debug (2/20/14 4:27:09 PM)] RunSetup: parent of jar/classes: C:/Users/ghowell/Documents/SikuliX/
[debug (2/20/14 4:27:09 PM)] RunSetup: RunningJava: Java 7 (amd64) 1.7.0_51-b13
[debug (2/20/14 4:27:09 PM)] RunSetup: ... could not detect whether Sikuli is used the first time on this system

--1. currently when rerunning setup, it is recommended, to delete everything in the setup folder (C:\Users\ghowell\Documents\SikuliX in your case) except sikuli-setup.jar and runSetup.cmd.
Then run setup again if needed using runSetup.cmd.

--2. to get more debugging info:
- in command line window go to C:\Users\ghowell\Documents\SikuliX
- use runIDE.cmd -d 3 -c
- the log output should tell you, what Sikuli has found about your monitor configuration.

I have some more interesting things to point out from my investigation. To give you an idea of my setup, I have a Dell laptop on a docking station and to the right of it I have a Samsung BX2440 monitor which is set as my primary display. I thought maybe it was the issue with the laptop being one of the displays, so I borrowed a monitor from another computer (a Dell something square screen) and made it the second display, closing the laptop lid. The samsung is plugged into monitor 1 and the Dell into monitor 2 on the docking station. At first I kept the samsung as the primary and tried the program. Again it sees the samsung as both screen(0) and screen(1) and behaved as before. But when I made the dell the primary display I got some weird results. The red highlight for screen(0) started on the far right edge of the Dell monitor and extended halfway across the samsung. The red highlight for screen(1) was around the whole screen of the Dell monitor, but it had shifted the left 3/4 of the screen up about the height of the start bar at the bottom. And then when my program did the selectRegion on screen(1) it only allowed me to select on the Samsung and what was weird is that it had shifted the contents of the Samsung screen over to the right about 2/3 of the screen so the left 2/3 was black and the right 1/3 showed 1/3 of the contents of the display. It said "Please select the region" in the black part but only allows me to actually select something over in the right 1/3 of the screen. And then the program crashes on the observe statement because it says
[error] Region(2560,225,639,358) outside any screen - subsequent actions might not work as expected
[error] Region(2560,225,639,358) outside any screen - subsequent actions might not work as expected
[error] script [ *Untitled ] stopped with error in line 18
[error] java.lang.NullPointerException ( java.lang.NullPointerException )

Could it be a display driver interaction thing? My display driver is an NVIDIA Quadro K2100M.

ok, just got your other response and I ran from the command line with debug like you mentioned. It tells me this:

[debug] Screen: initScreens: basic initialization (2 Screen(s) found)
[debug] *** monitor configuration (primary: 0) ***
[debug] Screen 0: S(0)[0,0 1280x1024]
[debug] Screen 1: S(1)[1280,-56 1920x1080]
[debug] *** end monitor configuration ***

That was with the two physical monitor configuration (no laptop screen) and the Dell as the primary display. So I noticed the negative Y position on screen(1). So I went back into my screen config and made the tops of the monitors line up. Now it shows:
[debug] Screen 0: S(0)[0,0 1280x1024]
[debug] Screen 1: S(1)[1280,0 1920x1080]

And everything works beautifully as expected!!! I think that was the culprit. It doesn't handle the negative coordinates well at least for Y. I then tried flipping the monitors again, making the one on the right the primary (the samsung 1920x1080) and the Dell the secondary. That made screen(1)s position -1280,0 but it all still behaved exactly as expected. I think it just doesn't like a negative Y value. What do you think?

Thanks for the detailed information and and for not giving up.

I will check the influence of a negative Y value on the monitor config in Sikuli.

all the best.

Thank you but I'm sorry to bug you with one more thing I noticed about it. So my Screen(0) is my main display on the right and my screen(1) is on the left. If I tell it to grab a region on screen(1), it correctly allows me to draw a region over there now (since I've lined up my monitors correctly). But when it actually does an observe in that region and sees something change, what it actually highlights is the correct spots in screen(1) but it fills the highlights in with screen images from screen(0). I believe that is a bug.

def changed(event):
        for ch in event.changes:
                ch.highlight() # highlight all changes
        sleep(1)
        for ch in event.changes:
                ch.highlight() # turn off the highlights
scr=Screen(1)
r = scr.selectRegion("Please select the region")
    # any change in r larger than 50 pixels would trigger the changed function
r.onChange(50, changed)
r.observe(10, background=False)

r.stopObserver()

If I were to venture a guess, since my default system SCREEN is 0 and I'm asking for a region over on screen(1), when it actually calls the function "changed" passing it the region coordinates, it uses those coordinates and highlights the appropriate elements with the same coordinates on screen(0) because that's the default instead of the using the screen that was passed. Is that a possibility?

if you are willing to test with a preview of 1.1.0-Beta1 and are able to download something from DropBox, then send me a mail silently to my mail at https://launchpad.net/~raimund-hocke and I will send you a link, where you can download a zip file for testing the IDE 1.1.0-Beta1.

Otherwise I will come back on that in 3 weeks (I am just off for a vacation ;-).

Hello,

When you modify the OS screen settings, you have to restart Sikuli.
Try to do your tests again and tell me if it's better.

I try on my Win7 with two screens in different positions, without find any trouble.

Regards.

Thank you for all the responses.

So it appears to find the screens correctly like you said and maybe it had something to do with restarting (although I thought I did that several times before), but the problem really isn't with the screen location and highlight now as much as it is with the way it observes and does actions over on Screen(1).

Here's what I'm seeing:

[log] highlight S(0)[0,0 1920x1080] E:Y, T:3.0 for 3.0 secs
[debug] Screen.capture: java.awt.Rectangle[x=0,y=0,width=1920,height=1080]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=0,y=0,width=1920,height=1080]
[log] highlight S(1)[-1920,-120 1920x1200] E:Y, T:3.0 for 3.0 secs
[debug] Screen.capture: java.awt.Rectangle[x=-1920,y=-120,width=1920,height=1200]
[debug] RobotDesktop: captureScreen: on 1 using java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
[debug] Screen.capture: java.awt.Rectangle[x=-1920,y=-120,width=1920,height=1200]

That part is working fine. It draws the red box around each screen correctly. Now, I have the following code to select a region over on screen(1):
scr=Screen(1)
r = scr.selectRegion("Please select the region")

[debug] CapturePrompt: Please select the region
[debug] ScreenUnion: getIdFromPoint: (1295, 87) as (-625, -33) in (-1920, -120,3840, 1200) on 1
[debug] CapturePrompt: started at (1295,87) as L(-625,-33) on 1
[debug] CapturePrompt: finished at (1905,841) as L(-15,721) on 1
[debug] CapturePrompt.close: freeing resources

That seems to capture the screen correctly on Screen(1)

But now I have code to observe changes in that region (of course my function is defined up at the top of the file):
def changed(event):
        #popup("something changed ")
        for ch in event.changes:
                ch.highlight() # highlight all changes
        sleep(1)
        for ch in event.changes:
                ch.highlight() # turn off the highlights

r.onChange(50, changed)
r.observe(10, background=False)

What I'm seeing is that it correctly observes a change in the region over on Screen(1) where I selected, but when it highlights and shows the contents, it is showing the contents of the equivalent region over on Screen(0) (i.e. it takes the contents of Screen(0) in that equivalent region and displays it in the highlighted area over in Screen(1)). And the other thing I tried the other day was to do a "click" or "rightclick" operation over in that region on Screen(1) and it actually did the click operation at the equivalent coordinates over on Screen(0)

[debug] Region: observe: starting in R[-625,-33 610x754]@S(1) for 10.0 seconds
[debug] SikuliEventManager: resetting observe states for R[-625,-33 610x754]@S(1)
[debug] SikuliXFinal: add observer: now running 1 observer(s)
[debug] Screen.capture: java.awt.Rectangle[x=-625,y=-33,width=610,height=754]
[debug] RobotDesktop: captureScreen: on 1 using java.awt.Rectangle[x=1295,y=87,width=610,height=754]

[log] toggle highlight M[-624,-32 555x656]@S(S(0)[0,0 1920x1080]) S:-1.00 Center:-347,296: true
[debug] Screen.capture: java.awt.Rectangle[x=1293,y=85,width=561,height=662]
[debug] RobotDesktop: captureScreen: on 0 using java.awt.Rectangle[x=1293,y=85,width=561,height=662]
[log] toggle highlight M[-624,-32 555x656]@S(S(0)[0,0 1920x1080]) S:-1.00 Center:-347,296: false
[debug] observe: update exit: ret: true
[debug] Screen.capture: java.awt.Rectangle[x=-625,y=-33,width=610,height=754]
[debug] RobotDesktop: captureScreen: on 1 using java.awt.Rectangle[x=1295,y=87,width=610,height=754]

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of obiwan-92
Sent: Wednesday, March 12, 2014 9:27 AM
To: Howell, Greg
Subject: Re: [Question #245030]: Screen(1) doesn't seem to work.

Your question #245030 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/245030

obiwan-92 requested more information:
Hello,

When you modify the OS screen settings, you have to restart Sikuli.
Try to do your tests again and tell me if it's better.

I try on my Win7 with two screens in different positions, without find any trouble.

Regards.

--
To answer this request for more information, you can either reply to this email or enter your reply at the following page:
https://answers.launchpad.net/sikuli/+question/245030

You received this question notification because you asked the question.

Ok. Thanks a lot.

Now, it's all clear. :-)

I will investigate this afternoon.

Hello.

The bug is confirmed.
It will probably resolve for the next version.

Thanks a lot to finding it. ;)

Regards.

You're welcome. And thanks for jumping on it so fast. So I only have 2 questions: 1) When do you predict the next version becoming available, and 2) can you verify that you've tested this no only for the highlight operation over on screen(1) but other operations like click and rightclick? I just wanted to make sure because those were other bugs that I saw and wasn't sure if they were related to the same root cause as the highlight issue.

Thanks,
Greg

Hello,

I can't tell you when the next version will come.
Sikuli is an open source application, every one can develop it.
Currently, there is only one people in development : RaiMan.
I believe he's currently in holidays.

For now, I'm just help the user of Sikuli to use the application and report the bugs.
In the few weeks, I will start to work on a recorder which be add to the IDE of Sikuli.

Right now, I don't have the time to correct your bug. Job troubles !! :'(

I have check the click, rightClick and hover functions. They work fine !!!
The problem is just of the capture. For some reason, the highlight capture an image from the Screen 0 and not the Screen 1. For me, it's just a minor bug and it's probably why we find it only now.

Again, I'm sorry I can't help you more. I will try to change that. ;)

Best regards.

Thanks for the reply. No need to be sorry. I was just curious on the timing. Glad to hear those other functions worked in your testing. I'll just look forward to the next build to try them out and do some work arounds in the mean time. I'll go ahead and close out this ticket.

Thanks again for the efforts