[EXAMPLE] XBMCFlix-o-mate Automating Netflix Silverlight Controls

---1. Thanx
... for that really interesting example.
The documentation and presentation of your work is outstanding.

---2. Installation of Sikuli and Java 6

on Windows 32-Bit systems, the program folder (standard installation folder) is:
C:\Program Files\

on Windows 64-Bit systems, there are 2 folders:
C:\Program Files\ for 32-Bit software
C:\Program Files (x86)\ for 64-bit software

In your ReadMe file this is not clear, since you mix up these folders:
These programs will install to c:\Program Files\Sikuli and c:\Program Files (x86)\Java respectively.

---3. Sikuli bug 729260
This is not a bug, it was the user's misunderstanding of what constant references to objects mean in Python.

The SCREEN variable is a constant reference to Screen(0) (the primary screen). Since the monitor preferences are only evaluated at script start, using SCREEN instead of Screen(0) only avoids additional screen objects.

#fix for Sikuli bug #729260
xScreen = SCREEN.x
yScreen = SCREEN.y
wScreen = SCREEN.w
hScreen = SCREEN.h
regionScreen = (xScreen,yScreen,wScreen,hScreen)
hoverPoint = getCenter()

This block currently does not really make sense, since regionScreen is not used anywhere in your script and the zScreen are the same as SCREEN.z and do never change while the script runs.

Since you use setROI() to restrict the search region in your hot key handlers, the searches are always done on Screen(0) (SCREEN).

A more flexible solution would be to define the main areas (ControlBarRegion, NextEpisodeRegion) once in you script as global variables and then use region.method()

e.g. in toggleNetFlixFullScreen
ControlBarRegion.click("fullscreen.png")

This would allow, to add multi monitor support easily to the script.

Naming hoverPoint as screenCenter would make things clearer, when reading the code.

---4. Handling FindFailed exceptions
It might be, that you have foreseen every eventual odd situation (I cannot judge), but if you did not, it is very hard to find reasons for the script's odd behavior because of "image not found" situations.

You switched of the FindFailed handling globally:
setFindFailedResponse(SKIP)

This might be, because you want to use the return value of find() in your workflow decisions.

The recommendation is, to use exists() for situations, where you have to decide how to proceed and let FindFails happen, where you did not expect them.

example:
# from netFlixLaunch

    netFlixLoading = wait("netflixlogo.png",8)
    if netFlixLoading:

better:
    if exists("netflixlogo.png",8):

It does the same, but you can leave FindFailed exceptions to be raised.

There are more examples throughout the script.

---5: repetitive usage of setROI()
When using setROI(), the ROI for the object SCREEN is set to the given region and it is maintained until it is set to another region or reset to the whole screen.

So if you call setControlBarRegion at the beginning of a workflow, it is not needed again, if you are targeting the same ROI.

e.g. in netFlixLaunch
the lines 21 and 29 are not necessary

But as already mentioned above: I would prefer to use region.method() instead of setRoi(); method()

---6: type("O", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL)
You trigger a hot key inside your script, to run the handler.
-1. if the user would change the hot key, he would have to go into your code, to adapt this.
-2. you do not need that, just use: netFlixLaunch(None) to run your Netflix startup stuff inline

---7: using, what is already found

e.g. in goNextEpisode

    nextEpisodeControl = find("nextepisode.png")
    if nextEpisodeControl:
        setNextEpisodeRegion()
        click("nextepisode.png")

better:

    if exists("nextepisode.png"):
        click(getLastMatch())

More examples in the script

---8: setAutoWaitTimeout(.1)
... does not make sense.
A GUI does not behave the same in the time or on different machines. The default of 3 seconds is a good standard value for the average situation. There are many people out there, who set the AutoWaitTimeout to a high value (e.g. 60 seconds), to be on the safe side for most situations. This does not harm, since Sikuli's standard maximum is 3 search trials per second, and the search is finished, when the visual object is found.
So setting the value to .1, restricts every search to one trial, which might not be enough on other machines. And it does not speed things up, it only increases the risk off odd behavior, especially in the combination with setFindFailedResponse(SKIP).

The only situation where one might want to restrict to one search trial:
You know, that something MUST be there and you want to do something, if it is not.
For these cases we have:
if exists(some_image, 0): # only one search
    print "it is as expected"
else:
    print "uuuups, what's the matter"

---9:
global subtitleToggleFlag

To avoid this Python specific situation, you might use the Settings class, to store your own settings:

Settings.mySubtitleToggleFlag = 1

        if Settings.mySubtitleToggleFlag==1:

BTW: if a setting only has 2 values, it is better to use the boolean True/False:
Settings.mySubtitleToggleFlag = False
        if not Settings.mySubtitleToggleFlag:

this is an example question

1) Thank YOU for the comprehensive feedback!

2) I use the "\Program Files\" directory pretty much exclusively when I talk about where things get installed. I feel at this point in time that anyone using a x64-bit system should know that non 64-bit programs install to the (x86) Folder So, I was just giving a quick overview. As I pointed users to your site to get Sikuli, and let them know that full instructions are there. I'll re-read the doc and either clarify or remove that part.

3) I think that section should have been removed. When I was initially having a problem with my debug issues (from the prior thread) I reworked things a bit. One of the things I did was to never use the full screen as my search region. Previous to this I was setting my ROI back to the whole screen, but realized that was unnecessary. I didn't however remove the code which generated that regionScreen for me. As far as the hoverPoint = getCenter(), do I need to define a region for that - or if I just put it early in the script prior to setting any specific region it will automatically use the center of the screen resolution?

I think I understand your preference for region.method as you describe, and will look to that again on next revision.

4) setFindFailedResponse(SKIP) vs. exists - I see your point here. I may have actually had exists() prior to reworking some of this to try other methods when I had my debug problem. I'm not sure it makes much of a difference to me here though. The behavior of the GUI is pretty expected, and if the image I want to find and act on isn't there, I simply want it to time out within a second or so and await another keypress. At this point in time, I don't see any exception that I would want to act on, since I don't know what else I could possibly do. The minute that becomes an issue however I will take this into consideration.

Is the timeout for an Exists controlled the same way as for a Find?

5) setROI - yes i realize the multiple calls were unnecessary, but I put them there as a safeguard in case additional code changes were inserted, this would make sure I was ALWAYS targeting the correct region on a specific click. This definitely was one of the things I changed during my debug issue because I wanted to make 100% certain. I feel they don't really hurt from either an overhead or readability perspective, but they are admittedly redundant. I do believe that the region.method as you suggest will be preferable though.

6) netFlixLaunch(None) - thanks...this was one of those areas that i wasn't sure how to adapt the code without a specific example. I didn't know how to call an event subroutine without the event!

7) getLastMatch - that seems to make sense, I don't think I found that method till later on, and was a bit confused to its purpose. It does seem that it will streamline a bit, so I will attempt those changes.

8) I think that situation you describe as the "only situation" is what I am looking for. One of my biggest complaints about Sikuli is the CPU overhead when searching for an image. These spikes can be disruptive, especially with my app when watching video. My perspective was that the image SHOULD always be there when the user presses the hotkey. If it is not, I don't believe waiting another 2-3 seconds is going to make a difference there in terms of the image appearing. Perhaps I am still not understanding your explanation, but I have found that the longer it is waiting around means the longer the CPU is searching for the image and that utilization maxes the processor on some machines. In my tests on two quite different machines, the clicks seem to always take place when expected. So, when you say it does not "speed things up" that is correct if I need to wait around for something to be clicked, but at this point I would rather it simply time out immediately if it can't fulfill the request rather than eat up CPU cycles - even for a few seconds.

9) I looked over the Settings class in the Sikuli doc. Are you saying that I can create any Settings.xxxxx class and reference it globally throughout the script? Is this a feature of Python in general or Sikuli? That certainly will make things easier...as I needed to figure out why my variable wasn't working properly until it was globally declared.

Unfortunately the Netflix plugin that I wrote this to assist has been dead in the water for a few months as the developer is MIA. The new version of XBMC was just released and the Netflix plugin is not listed in the repository for the new version because of this. So, I think i won't get a lot of user feedback until this is addressed, or at least it will take some time for word to get out there. There was previously another solution to do what my script does, but based on my research it no longer worked since Netflix changed their plugin last year. Apart from that, I think people have just been using AHK or AutoIt scripts to get their movie into full screen, but that would have to be customized based on screen resolution and didn't provide for the other features I included. In any event, the time I spent is well worth it to me, as I will be using it on a daily basis.

thx!

--1. ok

--2. ok

--3. ok
- hoverPoint = getCenter()
getCenter() returns the center pixel of the current ROI region of SCREEN which is Screen(0)

- showcase
print SCREEN
print "Bounds:", getBounds()
print "ROI:", getROI()
print "Center:", getCenter()

print "\n**** ROI 0,0,100,100"
setROI(0,0,100,100)
print SCREEN
print "Bounds:", getBounds()
print "ROI:", getROI()
print "Center:", getCenter()

print "\n**** ROI reset to screen"
setROI(getBounds())
print SCREEN
print "Bounds:", getBounds()
print "ROI:", getROI()
print "Center:", getCenter()

prints
Screen(0)[0,0 1920x1200] E:Y, T:3,0
Bounds: java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
ROI: java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
Center: (960,600)

**** ROI 0,0,100,100
Screen(0)[0,0 1920x1200] E:Y, T:3,0
Bounds: java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
ROI: java.awt.Rectangle[x=0,y=0,width=100,height=100]
Center: (50,50)

**** ROI reset to screen
Screen(0)[0,0 1920x1200] E:Y, T:3,0
Bounds: java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
ROI: java.awt.Rectangle[x=0,y=0,width=1920,height=1200]
Center: (960,600)

if you want to use the returns of getBounds() and getROI() as regions, you have to cast them to a Region:
screenCenter = Region(getBounds).getCenter()
ALWAYS returns the center pixel of the screen, no matter what ROI is set.

--4. generally ok, but as an example might lead people the "wrong" way

- Is the timeout for an Exists controlled the same way as for a Find?
exists() is better compared to wait(), which is a find with the possibility for a specific timeout.

so using the standard:
wait(image) # waits 3 seconds
wait(image, 10) # waits 10 seconds
wait(image, 0) # only one search

exists(image) # waits 3 seconds
exists(image, 10) # waits 10 seconds
exists(image, 0) # only one search

to make decisions with wait, you have to generally ignore FindFailed exceptions (as you do) or wrap it into ugly try:except:
So always recommend, to use exists() in all cases, where you want to use if:else:, in all other cases use find() or wait().

--5. ok, see 3.

--6. ok

--7. ok

--8. CPU usage
Yes this is sometimes a problem.
Internally Sikuli searches max 3 times a second for something. With the average searches, you have about 0.5 to 1 second search time (most people do not restrict the search region at all or not enough. In the consequence Sikuli is searching all the time and it is plain CPU usage and the java process gets all it can get.
If it is a problem, you can play with the waitScanRate (e.g. only search every 1.5 seconds).
But the real optimizer is restricting the search region as much as possible.
My first search always is at least restricted to the app window. Then I use every chance to restrict further and try to click on positions relative to matches I already have.
My average search time per find operation is around 0.3 seconds and lower with some scripts.
Since I add some short wait()'s for the GUI reaction, I do not have any problems within CPU usage.

--9. the Settings class exists in Sikuli already. So what you can do is add additional attributes (variable/values), that you want to use globally. You could do this with every global persistent object (like e.g. SCREEN).
I already had solutions using observe(), where I added variables to the region, to communicate between main and handler.
To avoid name conflicts, I always precede these vars with my...

Revised code is below. I believe I have taken into account all your suggestions, let me know if you think further optimization or clarification is necessary.

The only question remaining from above is your suggested code:
screenCenter = Region(getBounds).getCenter()

You can see I have that commented out below and replaced with a simple getCenter() because when using your suggested code I get the following:

"TypeError: org.python.proxies.sikuli.Region$Region$0(): 1st arg can't be coerced to org.sikuli.script.Region, java.awt.Rectangle"

Perhaps I misunderstood your example...?

Thanks!

----CODE START---

#Title: XBMCFlix-o-mate
#FileName: XBMCFlix-o-mate.sikuli
#Version: 1.1beta
#Author: bengalih
#
#Summary: A Sikuli script to automate Netflix controls within
# XBMCFlix, a plugin for XBMC.
# This script must be used in conjuntion with
# XBMCFlix-o-mate.au3 AutoIT script.

#Used on first launch of Netflix to bring into full screen mode
#Also used after switching episodes
def netFlixLaunch(event):
    if debug == 1:
        print("debug: in netFlixLaunch (Ctrl+Alt+Shift+O)")
    #Wait 8 seconds for launch of new Video, otherwist timeout to prevent CPU usage
    if NextEpisodeRegion.wait("netflixlogo.png",8):
        if debug == 1:
            print("debug: loading movie")
        #Wait up to 60 seconds for movie to buffer
        if NextEpisodeRegion.waitVanish("netflixlogo.png",60):
            if debug == 1:
                print("debug: loaded movie")
            #Set movie to full screen if not already
            if not ControlBarRegion.exists("exitflag.png",1):
                ControlBarRegion.click("fullscreen.png")
            if debug == 1:
                print("debug: full screen action!")
        else:
            if debug == 1:
                print("debug: timed out waiting for Netflix logo to vanish")
    else:
        if debug == 1:
            print("debug: timed out waiting for Netflix logo to appear")

#Toggles full screen
def toggleNetFlixFullScreen(event):
    if debug == 1:
        print("debug: in toggleNetFlixFullScreen (Ctrl+Alt+Shift+F)")
    click(hoverPoint)
    if ControlBarRegion.exists("fullscreen.png",1):
        click(ControlBarRegion.getLastMatch())
        hover(hoverPoint)

#Advances to next episode either during show or at end screen
def goNextEpisode(event):
    if debug == 1:
        print("debug: in goNextEpisode (Ctrl+Alt+Shift+N)")
    #Check if episode is over and Next Episode prompt is on screen
    if NextEpisodeRegion.exists("nextepisode.png",1):
        click(NextEpisodeRegion.getLastMatch())
        netFlixLaunch(None)
    #If in middle of episode use skip button instead
    else:
        click(hoverPoint)
        if ControlBarRegion.exists("skipbutton.png",1):
            click(ControlBarRegion.getLastMatch())
            netFlixLaunch(None)
        else:
            #Toggle out of full screen mode to find skip button
            if ControlBarRegion.exists("fullscreen.png",1):
                click(ControlBarRegion.getLastMatch())
                if ControlBarRegion.exists("skipbutton.png",1):
                    click(ControlBarRegion.getLastMatch())
                    netFlixLaunch(None)
    hover(hoverPoint)

#Goes back one episode
def goPreviousEpisode(event):
    if debug == 1:
        print("debug: in goPreviousEpisode (Ctrl+Alt+Shift+B)")
    click(hoverPoint)
    if ControlBarRegion.exists("prevbutton.png",1):
        click(ControlBarRegion.getLastMatch())
        netFlixLaunch(None)
    else:
        #Toggle out of full screen mode to find back button
        if ControlBarRegion.exists("fullscreen.png",1):
            click(ControlBarRegion.getLastMatch())
            if ControlBarRegion.exists("prevbutton.png",1):
                click(ControlBarRegion.getLastMatch())
                netFlixLaunch(None)
    hover(hoverPoint)

#Toggles English subtitles on and off
def toggleEnglishSubtitles(event):
    if debug == 1:
        print("debug: in toggleEnglishSubtitles (C+A+S+T)")
    click(hoverPoint)
    #If Subtitles exist, create smaller region for search
    subtitleControl = ControlBarRegion.exists("subtitles.png",1)
    if subtitleControl:
        click(ControlBarRegion.getLastMatch())
        subtitleOptions = subtitleControl.nearby(100)
        subtitlesEnglish = subtitleOptions.find("subenglish.png")
        subtitlesOff = subtitleOptions.find("suboff.png")
        if not Settings.mySubtitlesOn:
            click(subtitlesEnglish)
            Settings.mySubtitlesOn = True
        else:
            click(subtitlesOff)
            Settings.mySubtitlesOn = False
        ControlBarRegion.click("subtitles.png")
        hover(hoverPoint)

#sets the scan region to the lower 20% of the screen
def setControlBarRegion():
    regionX = 0
    regionY = (SCREEN.h/10)*8
    regionW = SCREEN.w
    regionH = SCREEN.h-regionY
    global ControlBarRegion
    ControlBarRegion = Region(regionX,regionY,regionW,regionH)

#sets the scan region to the middle area of the screen
def setNextEpisodeRegion():
    regionX = (SCREEN.w/10)*3
    regionY = (SCREEN.h/10)*3
    regionW = (SCREEN.w/10)*4
    regionH = (SCREEN.h/10)*4
    global NextEpisodeRegion
    NextEpisodeRegion = Region(regionX,regionY,regionW,regionH)

debug = 1
Settings.MoveMouseDelay = 0
Settings.mySubtitlesOn = False

#sets the hover point for the mouse to the center of the screen
#screenCenter = Region(getBounds).getCenter()
screenCenter = getCenter()
hoverPoint = screenCenter

#initializes the two scan regions
setControlBarRegion()
setNextEpisodeRegion()

#User Defined hotkeys in block below:
#If changing the netFlixLaunch key away from "O", be sure to
#also update the AutoIt script!
####################################################################
Env.addHotkey("O", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL, netFlixLaunch)
Env.addHotkey("F", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL, toggleNetFlixFullScreen)
Env.addHotkey("N", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL, goNextEpisode)
Env.addHotkey("B", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL, goPreviousEpisode)
Env.addHotkey("T", KeyModifier.ALT+KeyModifier.SHIFT+KeyModifier.CTRL, toggleEnglishSubtitles)
####################################################################

#Main subroutine
while True:
    wait(1)

--- screenCenter = Region(getBounds).getCenter()
Sorry, a typo :-(
screenCenter = Region(getBounds()).getCenter()

In your case this is not needed any more.
In cases where you work with setROI(), this is the way to reference the whole screen for Region methods, that otherwise would use the current ROI

--- one more thing
These settings are typical usage of functions with return values, so you do not need any globals for that.

#sets the scan region to the lower 20% of the screen
def setControlBarRegion():
    regionX = 0
    regionY = (SCREEN.h/10)*8
    regionW = SCREEN.w
    regionH = SCREEN.h-regionY
    return Region(regionX,regionY,regionW,regionH)

#sets the scan region to the middle area of the screen
def setNextEpisodeRegion():
    regionX = (SCREEN.w/10)*3
    regionY = (SCREEN.h/10)*3
    regionW = (SCREEN.w/10)*4
    regionH = (SCREEN.h/10)*4
    return Region(regionX,regionY,regionW,regionH)

.... and then
#initializes the two scan regions
ControlBarRegion = setControlBarRegion()
NextEpisodeRegion = setNextEpisodeRegion()

and since both functions share similar code, one should make one out of it:
(the DRY design pattern: Don't repeat yourself ;-)

# Set a region on the screen by relative margins ignoring ROI
def setRegion(top=0, bottom=0, left=0, right=0):
    scr = Region(getBounds())
    regionX = int(scr.w*left)
    regionY = int(scr.h*top)
    regionW = scr.w - int(scr.w*left) - int(scr.w*right)
    regionH = scr.h - int(scr.h*top) - int(scr.h*bottom)
    return Region(regionX,regionY,regionW,regionH)

.... and then
#initializes the two scan regions
ControlBarRegion = setRegion(0.8)
NextEpisodeRegion = setRegion(0.3, 0.3, 0.3, 0.3)

--- one more thing more ;-)
# in toggleEnglishSubtitles
    subtitleControl = ControlBarRegion.exists("subtitles.png",1)
    if subtitleControl:

... could be
    if ControlBarRegion.exists("subtitles.png",1):

Your version only makes sense, if you could use subtitleControl again later in your code,
e.g. instead of
      ControlBarRegion.click("subtitles.png")
use
      click(subtitleControl)

if the search would evaluate to the same match as with the exists() already

--- one last thing: multi monitor
I do not know, wether this is relevant: multi monitor configurations.
Is the XBMC window always on the primary screen? Or can you run it on other monitors of you have them?

You might evaluate the app window and use Region.getScreen(), to know the monitor, it is running on.

I like the change to the screen region routines.... I do use the DRY approach in my VBscripting, but wasn't sure exactly how to pass those in the Python subroutine...your example helped with that.

Regarding your quote:
"--- one more thing more ;-)
# in toggleEnglishSubtitles
    subtitleControl = ControlBarRegion.exists("subtitles.png",1)
    if subtitleControl:

... could be
    if ControlBarRegion.exists("subtitles.png",1):

Your version only makes sense, if you could use subtitleControl again later in your code,
e.g. instead of
      ControlBarRegion.click("subtitles.png")
use
      click(subtitleControl)"

I do use subtitleControl again directly following that where I set "subtitleOptions = subtitleControl.nearby(100)"
is there a more efficient way to do that, or did you just overlook that?

Multi-monitor may be necessary for some fringe users...I was not too worried about supporting it, but if it is easy enough to account for, I might as well. I looked over some of the documentation on Region.getScreen and the bounds() and am still a bit confused what I would have to do to implement this.

Up to this point, all window activation is being handled by the AutoIt script as I couldn't find a way for Sikuli to detect windows without bringing one into focus. So would I need to use the .App class to somehow find the Netflix window and then use a .getScreen? I am unsure at what points I need to define this? Can I just do it once within the def setRegion?

Part of the problem is that the way not only the new way IE deals with Windows (multiple hidden instances of IE) but also something I have not yet figured out seems to happen when NetFlix goes into Fullscreen. It seems that there are separate multiple windows open for IE and Silverlight (even though silverlight is just running as a plugin to IE). It is hard to track down because like Heisenberg...when Netflix is in true Full Screen I can't gather info from anything...and as soon as I switch to a window finder tool, or task manager, I take Netflix out of true fullscreen!

I also don't have a multimonitor setup to test, but if there a few simple commands I can put in for a best-effort for those that might be using it, I wouldn't mind.

thanks.

--- I do use subtitleControl again directly following ...
yes, I have overseen that.
so forget it ;-)

--- multi monitor
When you search something in a multi-monitor environment, you have to specify the screen region, when searching the whole screen and it is not the primary monitor.

find(something) # searches on SCREEN (= Screen(0))
SCREEN1 = Screen(1)
reg = SCREEN1.find(something) # will search on screen 1
reg.below() # now is a region on screen 1 (world coordinates with ref to (0,0) upper left of SCREEN)

This means, in a multi monitor environment only the full screen searches have to specify the screen (wherever you get this information from).

So does this mean that I need to:

a) Find out what screen the Netflix window is running on? I assume somehow with the App class...but I don't see how this is easily done.

b) Create a new screen object for that screen? Seems easy enough: netFlixScreen = Screen(#) once you have the # from above.

c) Modify my def setRegion somehow to set the scr = Region(getBound()) for the new screen object? I have read over the class Screen documentation several times and quite honestly it doesn't make a lot of sense to me. As I mention I am not a programmer by trade and there isn't a single example of how to use any of the class Screen objects.

I also don't understand the logic behind the coordinate system described - it seems overly complicated than just assigning a new set of base for each screen. I also wonder whether this method causes problems if the other monitors have different resolutions? I can't tell because I just can't make sense of it.

Wouldn't the coordinate system require something more complicated than:

def setRegion(top=0, bottom=0, left=0, right=0):

since the default coordinates for a given screen will not be 0, but some relative offset based on the monitor position?

This may be too complicated, I just can't tell since I have no examples to go off of.

Thanks.

a) There is always some research necessary, to get some structure into the window titles. But that is the easiest approach on Windows.
If you know the part of the window title, that selects it among others:
app = App("exclusive part of window title")
reg = app.window()

if more than one window contains the text:
app = App("exclusive part of window title")
for n in range(100):
    win = app.window(n)
    if not win: break
    # here we can check some aspects of the window,
    # to find out the right one
    # position, dimension, content

this all does not do any focus action.

b) yes

c) The coordinate system is based on how Java is handling multi monitor stuff:
- primary monitor: (0,0) top left (w1 x h1)
- secondary monitor:
 e.g. if positioned right of primary in the system settings, its top left coordinates are (w1, 0)
 e.g. if positioned left of primary in the system settings, its top left coordinates are (-w1, 0)
 e.g. if positioned above of primary in the system settings, its top left coordinates are (0, -h1)

and so on.

So if you know your monitor setup, you can say (second example):
reg = Region(-1000, 400, 100, 100)
and reg is a region somewhere on the second monitor.
This systematic is used with Matches and Locations too.

The Screen class itself is only an extension of the Region class, with some special aspects with respect to the physical monitor and the convenience, that in the IDE (Python level) you do not need to qualify the methods with a screen object, if you target the primary screen.

Before I look into it any further, I just want to qualify your quote:

"So if you know your monitor setup..."

Does this mean that there would not be a way to detect what a particular monitor setup is?

I mean, "getNumberScreens()" can return that I have 3 screens....but it doesn't know how these are positioned until we do some sort of check of each region based on these offset coordinates?

And we don't know what the offset coordinates are until we get the bounds of each screen object?

Sorry...but screen placement (to me) is a very visual thing, and without seeing visual examples of coordinates and how these are referenced in code, I just won't get it. Perhaps when your team's time allows the updating of the documentation to provide this I will be able to take another look.

As stated, I don't think it is going to be too important to users of my app anyway.

--- the updating of the documentation
I do not think, that anything needs to be added to the docs at:
http://sikuli.org/docx/screen.html#multi-monitor-environments

you even have a picture ;-)

I agree, that this might not be a current priority for you, but Sikuli has all the features, to handle the multi monitor situation correctly.

Well - it is your call of course. :)

I have told you previously that I am not a programmer by trade, but I hope at this point in the conversation you would not take me for a hopeless fool.

That being said, I have a very hard time deducing from the documentation, as well as your examples what would need to be done to implement this. I don't know what type of feedback you have had in the past, so I am simply giving you my experience:

If you have had positive feedback from 10 other people who also are not programmers by trade who had absolutely no problem figuring out multiple monitor usage, then I guess I'm the odd man out...

If you haven't have much feedback from other non-programmers - well then I give you my experience as feedback.
My suggestion would be to include at least one coding example on that page with a reference picture (as this is a very visual, coordinate oriented area).

As this feature isn't that necessary for me at this point I don't feel like spending significant time experimenting with syntax to determine what does and doesn't work (as I have had to do with several of the methods that didn't include example documentation).

The above is not a criticism. I find your documentation overall very good, and more readable than that of the Python docs themselves. I am just offering my experience with learning the language. The more accessible you make it, the more people you bring in, the less questions you have to answer (or at least stupid questions :)), and the more you foster learning in others.

--- hope you would not take me for a hopeless fool
If anything in my comments (I cannot see that from my view) made you feel that I am thinking this way, than I apologize for that.
In contrary I think, that you managed the situation perfectly with stuff, that is not at the level of a "hopeless fool".

--- needed knowledge for Sikuli
It is always a problem for someone with little experience in scripting (I do not talk about programming here), to manage more complex workflows, that consist of more than a stream of find/wait/click/type actions. Many new bees already have problems with implementing decision trees and/or loops. At least a basic understanding for a scripting language like Python is necessary. But if you are willing to, you might learn that with Sikuli.

--- understanding the basic concept of Sikuli
... is vital, if you want to use it effectively. It is a great convenience, that you can create a click(some_image) with the help of the command bar and that this automatically directed towards the primary screen, but this hides the complexity behind that. It really is: some_region.click(some_image).
So to help to understand that, you need examples, that differ fundamentally from the scripts, a newbee would create in the IDE, when starting with the IDE.

--- examples
Since Sikuli is a visual tool, setting up examples is very complex stuff. And in most cases, these examples cannot taken as is, since the images might not match in the users environment (getting FindFailed situations is very frustrating, if see it on the screen, but Sikuli does not).
So for a tool like Sikuli you need some interactive tutorial, that would help you to find a way.
The extension Guide was a start in that direction, but the development was stopped last year.

So in the concrete situation multi monitor, the only mentionable thing is, that you have to say:

Screen(1).find(some_image)

if you know, that something should be on the secondary screen.

--- making scripts that run on different systems/environments
If you try to do that, then you are already on an advanced scripting level (and for more complex things it should be called programming then (using modules, classes, image libraries, system specific stuff, ...)).

--- One last thing
I really dislike statements like
The above is not a criticism.

What else is it? This sounds like excusing yourself upfront for having your own sight on things, naming faults and suggesting alternatives. Criticism should always be positive and helpful, but it is needed to bring things forward. And I think your criticism is positive and helpful, so lets name it as such.

I wasn't inferring you thought I was a fool...I had hoped it was a rhetorical question as I am pretty confident I have shown some degree of competence with my initial script and taking your generous revisions into account :)

Regarding the "Criticism" statement: I feel there is a big difference between a criticism and a suggestion/having ones own views. A criticism would be that I thought the way you did something was poor/inefficient regardless of viewpoint - that I felt it held true for all situations - and that I though it was bad judgement/poor form. In this regard I have no criticism towards anything discussed. I tend to never even use the work criticize and in general dislike when others use it (e.g. "I just have one minor criticism..."). In most cases "suggestion" is more apt.

I do not back down or excuse myself in the least for having my own viewpoint - which I have conveyed above. What the Sikuli team chooses to do with that - who they choose to target, how they choose to layout the docs, etc - is fully their decision. And the decision not to cater (as much) to those who are not professionals is a perfectly valid decision - one that shouldn't be criticized. I don't know the Sikuli base - in my head I am making an assumption that your users fall into 3 major categories:

1) Professional developers who are most likely using Sikuli integrated with larger projects (like QA testing). These guys know what they are doing and get paid for it. Some of these users may also be playing with Sikuli for small projects of a personal nature.

2) Total new bees who are not interested in anything more than point-and-click. As I think you mentioned before people who are looking at a way to script games, etc.

3) The middle area - users like me who are not professional coders, but have some degree of knowledge in coding and syntax (personally my experience is with scripting langauges: original BASIC, batch programming, VB script, Mortscript, PowerShell, and a moderate degree of VB, some HTML/ASP, and various degrees of hacking at some more advanced stuff).

I'm not sure how the above actually breaks down into percentages. Personally, if I were tackling support/documentation I would just about disregard the #2 set. Mostly since I don't condone scripting to cheat at games - but also because I would expect someone to show some level of interest in the tool and the language and not just end-result. Someone else may not agree with my logic - thinking that the developer shouldn't care about the use of the tool (just creating the tool is their onus, not what others do with it), or that it is ONLY the end result that matters and not how someone get there.

On the other hand, I believe you get a lot of exposure by being a partly WYSIWYG development platform. I actually think you have done a good job with selling this. The very simple tutorials following the main Demo Video do a good job of teaching the IDE basics so that almost anyone can get running to make a simple point-and-click.

After those few examples though it seems to be straight to the documentation, which caters a bit more to the #1 set. I think that this is practical as the official docs should talk in programmatic terms and provide the high level technical detail (outlining the classes, methods, etc.). I do feel that one of those new bees jumping from the IDE to the docs (having no previous experience coding/scripting) is likely to be utterly lost. And, to be honest - I think that is OK - because I don't think it is your place to teach them coding.

I do though still feel that there is room in the docs (or perhaps a separate document/wiki) to assist those more in the #2 set. Of course, this suggestion is born of personal preference. Perhaps the #2 set is very small in your community and not worth the effort. Perhaps you find it more efficient to simply handle all the questions in this answers forum rather than try to proactively undercut them by offering more in the way of examples. Perhaps you simply don't feel supporting the #2 set is necessary. Whatever your viewpoint - it must have sound logic behind it and therefore I would not criticize it. I would simply offer my experience as you may have not had feedback in the past in this area.

I am always one for healthy debate - but I don't think alternative viewpoints should be viewed as criticism. My very holistic acting teacher in college was very keen on semantics when it came to things like this. Saying something like "You're a jerk" is not very conducive to discourse. Saying "When you do so-and-so it makes ME feel that you think I'm insignificant" is a non-accusatory way of letting others know how their actions effect you without criticizing them directly. Yes, it is all semantics to a degree, but one of the things that sets us apart from the apes is our use of language :)

As far as the technical on Multi-Monitor - perhaps I am just clouded by a block right now. I really did breeze through most of picking up Sikuli (by merit of the videos, docs, and your assistance). Short of knowing python and integrating that into my scripts, I believe I am utilizing most of the features in my trials. This area is putting up a bit of a brick wall and maybe I should just step back and re-visit it once the rest settles. If I have issues when I actually try to implement I will start a new topic with my specific problems.

As always, thanks for your time.

--- Regarding the "Criticism" statement:
I totally agree with all that you say.
In any case one should talk about things, that you think should be changed by other people, in a way, that does not hurt or sounds arrogant. The easiest way to do this is how your college teacher told you to do it.
On the meta level it is a question of culture or other social aspect, how you name it, when "criticizing" something/someone. I do not have any problems to name it criticism, since for me this is neutral. But naming it feedback might be accepted easier by most people.
Nevertheless, I think statements like "This is not criticism, but ...." are needless or even dangerous, when you give feedback in a positive way (like you do it). Untrusting people might escalate the discussion in the wrong direction and others might take it as an upfront excuse like I mentioned above (and take it as a sign for uncertainty).

One last thing, that is really inviting to comment:
You say: I tend to never even use the work criticize and in general dislike when others use it .
But then: Why do you say: The above is not a criticism. ??? ;-)

--- grouping Sikuli users
Very good classification.
I guess no one knows about the percentages.
I think, there are many out there in group 1), but they are rather quiet in this forum (little questions, some bugs, a few answers). Some of them talk about their testing solution somewhere else in the net.
The people from group 2) try to do some stuff with Sikuli and either win or loose. In some cases they ask "stupid" questions and in many cases they have problems to understand the answer (because already the question topic is beyond their scripting knowledge). These people would need tutorials and copy-paste examples, but that is rather complex with Sikuli.
I think the most people asking questions in this Q&A board belong to group 3). I have many cases, where it comes out, that they have tried different solutions/approaches sometimes for days, then finally decided to ask and are surprised about how easy the solution sometimes comes along. I think these people would need more intelligent/interactive tutorials. Nevertheless: I agree, that at some places in the docs more coding examples should be added.

--- game bots
I agree, that trying to automate a game (at least one where you play with others) is not very fair. But trying to cheat is as old as mankind ;-)
I think there is another valid aspect: Trying to make a game bot with Sikuli is the most challenging thing among all scenarios, since its GUI's are far more complex than with normal apps and scripting the rules and actions is a very demanding job. So if you finally succeed in getting it to work, you can be very proud.

Thanks for talking ;-)

All the best and always welcome with new questions.