[HowTo] type(somekey, KEY_META) inconsistent? -- use lowercase letters

Sorry for the noise - this seems like an operator error (?) ---

replace:

type("L",....)

with:

type("l", ....)

...now apps respond as expected.

a valuable HowTo

even noise and ?operator errors? documented here (yours is documented exemplarily :-) might help others.

You can optimize your script with respect to the wait()'s:

This will continue in the moment, the window is there.

browser = App("firefox3")
if not browser.window():
    browser.focus() # opens FF if not open
    wait(1)
type("l", KEY_META); wait(0.5)
paste("www.sikuli.org"); type(Key.ENTER)

I consequently use paste() to type text() and type() only for special keys.

sorry, too early in the morning:

again the script:

browser = App("firefox3")
while not browser.window():
    browser.focus() # opens FF if not open
    wait(1)
type("l", KEY_META); wait(0.5)
paste("www.sikuli.org"); type(Key.ENTER)

Understood - thank you for suggestion / optimization!

On Sun, Feb 6, 2011 at 2:56 AM, RaiMan <<email address hidden>
> wrote:

> Your question #144310 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/144310
>
> RaiMan posted a new comment:
> sorry, too early in the morning:
>
> again the script:
>
> browser = App("firefox3")
> while not browser.window():
> browser.focus() # opens FF if not open
> wait(1)
> type("l", KEY_META); wait(0.5)
> paste("www.sikuli.org"); type(Key.ENTER)
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Regards,
- Yarko

solution found by user himself

Ok - _this_ appears to be a bug (at least on OS-X):

browser.window() doesn't change value, so in the above while loop, it is stuck there forever...

Changing to something like:

browsername = "firefox3"
browser=App(browsername)
if not browser.window():
        browser.open() # opens if not open
        wait(5)
if not browser.window():
    popup("Houston, we have a problem opening the window")

else:
    popup("Success!")

the "if not...." always is executed.

the intention was, to either use an existing browser window or open the browser and wait until the window is ready.

the construct for this purpose has to be

browser = App("firefox")
browser.focus() # opens FF if not open
while not browser.window():
    wait(1)
type("l", KEY_META); wait(0.5)
paste("www.sikuli.org"); type(Key.ENTER)

works perfectly on my Mac.

On Tue, Feb 8, 2011 at 1:54 AM, RaiMan <<email address hidden>
> wrote:

> Your question #144310 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/144310
>
> RaiMan posted a new comment:
> the intention was, to either use an existing browser window or open the
> browser and wait until the window is ready.
>
> the construct for this purpose has to be
>
> browser = App("firefox")
> browser.focus() # opens FF if not open
> while not browser.window():
> wait(1)
> type("l", KEY_META); wait(0.5)
> paste("www.sikuli.org"); type(Key.ENTER)
>

Two notes:

- This too doesn't work (I seem to be in the while loop eternally; with
this construct, firefox never sees the text, and I can quit firefox, but
sikuli IDE never returns to focus - so I assume it's still in the loop; in
the previous construct you posted, kit would repeatedly open firefox: e.g.
 firefox would get no text; I would quit firefox, firefox would (in the
loop) get opened again... I had to force/quit sikuli in the small window I
had between firefox re-launching / gaining focus.

$ java -version
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261)
Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode)

How can I help debug this?
Also - what is the expected return from <app object>.window()?

Regards,
Yarko

>
> works perfectly on my Mac.
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Regards,
- Yarko

Absolutely right, the first loop was not very intelligent ;-)

But the last one absolutely works fine.

<app object>.window() (same as window(0)) contains the region of the app's frontmost window. This is None, if no window is there (this is the case, as long FF is starting up and when it is running (even if no window is open, FF opens a new one, if gets focus)).

I doubt it might have something to do with your "firefox3", since the standard name of FF when installed on Mac is Firefox.app. Just check with the task-monitor the name of the system process.

So I guess you have Safari on your machine also ;-)
Make a test with "Safari" instead of "firefox3" and I bet it works.

Raiman -

That's a reasonable guess, but unfortunately (or fortunately) wrong:

If doesn't matter if I open firefox (I renamed to firefox3, so no difference), safari, or chrome.
The behavior I see is this:
- the browser opens;
- nothing is "typed" / "pasted"
- I kill the browser; sikuli-ide shows (now) as current app, but still hidden.
- I META-Q sikuli-ide, and get a dialog saying "unsaved changes".
- If I "cancel" - there is some recovery problem, as keyboard input is now NOT connected to the sikuli-ide

I assume on Mac, /Applications/Sikuli-IDE.app/sikuli-ide.sh is what is run, so I have tried to modify, run with
/Applications/Sikuli-IDE.app/Contents/MacOS/JavaApplicationStub64
in place of
/Applications/Sikuli-IDE.app/Contents/MacOS/JavaApplicationStub
and that doesn't change behavior.

I've noticed this post by Catherine Devlin:

http://catherinedevlin.blogspot.com/2010/09/pdb-with-sikuli.html

... but this needs some modifications for debugging on the "mac install".

Like I said before, if you can give me a quick pointer to help debug, that would be helpful.
I can sort it out myself, or build from the tree (eventually) - but, alas, I'm trying to solve something
else, which I'm hoping to be able to use sikuli for!

BTW - wherever implemented, I will want to run on multiple O/S's (mostly Mac & Win) - and so far, I have also noticed that the "executable" ... well, isn't: to run on Windows, I need to install the windows sikuli, and also if I try to run in ViMWare (windows, running in virtual environment on mac) - it doesn't seem to work (last night, any run took me to the login screen), but that's a different debugging effort.

Regards,
Yarko

Really strange, I have exactly the same environment as you have and it works like a charm.

Currently the only difference is, that I am using a pre-version of Sikuli X-1.0rc2 (planned release 10th Feb), that I have bulild myself as a fully 64-Bit version from source.

But as far as I remember with X-1.0rc1 this worked for me too.

I have two monitors and am using Spaces.

The only problems I had with this loop was, when Safari or Firefox was running already and their windows where in different Spaces (the App.focus currently does not switch Spaces). but even in one Space it was no problem having Sikuli on Screen(0) and FF opening on Screen(1).

Try the following:
- everything on Screen(0)
- quit FF (no more open window)
- open Sikuli-IDE by double-clicking the app symbol

test 1: in IDE:
browser = App("Firefox")
browser.focus()
wait(5)

You should see FF start up, open an empty window and finally Sikuli-IDE should come back (partly hidden by FF). leave FF as is.

test 2: in IDE:
browser = App("Firefox")
browser.focus()
wait(1)
r = Region(browser.window(0))
print r
r.highlight(3)

before starting the script put e.g a Finder window in front of the FF window.

running: the FF window should come to front and for 3 seconds have a red frame around it. After that Sikuli-IDE should come back.

the print r should have produced something like:
Region[66,22 1682x1047]@Screen(0) E:Y, T:3,0
and may be the highlight produced a log message.

If this does not work (no more betting ;-), you should have a try with rc2 in a few days.

Comments on the other things:
- if I want to debug Sikuli scripts I use Netbeans with the Jython plugin. There the debugger works fine.
- Since a friend of mine needed a Win7 laptop, I gave him my MacBookPro/Bootcamp/Win7. So the next thing I will do is test Sikuli in this VM environments (again, with 0.10 there were many problems) with Parallels and the new Version of VirtualBox.

On Tue, Feb 8, 2011 at 11:22 AM, RaiMan <
<email address hidden>> wrote:

> Your question #144310 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/144310
>
> RaiMan posted a new comment:
> Really strange, I have exactly the same environment as you have and it
> works like a charm.
>
> Currently the only difference is, that I am using a pre-version of
> Sikuli X-1.0rc2 (planned release 10th Feb), that I have bulild myself as
> a fully 64-Bit version from source.
>
> But as far as I remember with X-1.0rc1 this worked for me too.
>
> I have two monitors and am using Spaces.
>

.... same here;

>
> The only problems I had with this loop was, when Safari or Firefox was
> running already and their windows where in different Spaces (the
> App.focus currently does not switch Spaces). but even in one Space it
> was no problem having Sikuli on Screen(0) and FF opening on Screen(1).
>

Haven't played w/ them in different spaces....

>
> Try the following:
> - everything on Screen(0)
> - quit FF (no more open window)
> - open Sikuli-IDE by double-clicking the app symbol
>
> test 1: in IDE:
> browser = App("Firefox")
> browser.focus()
> wait(5)
>

Yes - as I remember / expected (it's the while loop that hangs!) -
additionally, IDE console output is:
Sikuli vision engine loaded.
Text Recgonizer inited.
Mac OS X utilities loaded.
VDictProxy loaded.
EventManager loaded.
[log] App.focus Firefox3(0) #0
[log] openApp: "Firefox3"
[debug] close all ScreenHighlighter
[debug] close all ScreenHighlighter
[info] Exited with code 0

> You should see FF start up, open an empty window and finally Sikuli-IDE
> should come back (partly hidden by FF). leave FF as is.
>
> test 2: in IDE:
> browser = App("Firefox")
> browser.focus()
> wait(1)
> r = Region(browser.window(0))
> print r
> r.highlight(3)
>
> before starting the script put e.g a Finder window in front of the FF
> window.
>
> running: the FF window should come to front and for 3 seconds have a red
> frame around it. After that Sikuli-IDE should come back.

> the print r should have produced something like:
> Region[66,22 1682x1047]@Screen(0) E:Y, T:3,0
> and may be the highlight produced a log message.
>

Yes - correct: here is the IDE log:

[log] App.focus Firefox3(0) #0
[log] openApp: "Firefox3"
Region[127,29 1491x984]@Screen(0) E:Y, T:3.0
[log] highlight Region[127,29 1491x984]@Screen(0) E:Y, T:3.0 for 3.0 secs
[debug] close all ScreenHighlighter
[debug] close all ScreenHighlighter
[info] Exited with code 0

>
> If this does not work (no more betting ;-), you should have a try with
> rc2 in a few days.
>

It works.

I will try Netbeans (haven't used).

I may also see if I can try to build from repo, and see how that goes.

>
> Comments on the other things:
> - if I want to debug Sikuli scripts I use Netbeans with the Jython plugin.
> There the debugger works fine.
> - Since a friend of mine needed a Win7 laptop, I gave him my
> MacBookPro/Bootcamp/Win7. So the next thing I will do is test Sikuli in this
> VM environments (again, with 0.10 there were many problems) with Parallels
> and the new Version of VirtualBox.

> --
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Regards,
- Yarko

last try (just downloaded rc1, tried and this works):

b = App("Firefox")
b.focus()
while not b.window():
   wait(1)
Region(b.window()).highlight(2)

-- Netbeans: very tricky with rc1 on Mac. If you do not succeed, I know how it works.

-- building from source on Mac (especially 64 Bit): a few tweaks necessary

Both things I will finally document for rc2 next week. If you need help before: mail to: https://launchpad.net/~raimund-hocke.

On Tue, Feb 8, 2011 at 12:22 PM, RaiMan <
<email address hidden>> wrote:

> Your question #144310 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/144310
>
> RaiMan posted a new comment:
> last try (just downloaded rc1, tried and this works):
>
> b = App("Firefox")
> b.focus()
> while not b.window():
> wait(1)
> Region(b.window()).highlight(2)
>

heh - I forgot to rename "Firefox" to "Firefox3" for my install - and I had
to quite the IDE / cancel the quit caused the loss of keyboard focus (you
could probably reproduce with some invalid App name).

I _did not expect_ this to work, but it does! ...

IDE log:
Sikuli vision engine loaded.
Text Recgonizer inited.
Mac OS X utilities loaded.
VDictProxy loaded.
EventManager loaded.
[log] App.focus Firefox3(0) #0
[log] openApp: "Firefox3"
[log] highlight Region[127,29 1491x984]@Screen(0) E:Y, T:3.0 for 2.0 secs
[debug] close all ScreenHighlighter
[debug] close all ScreenHighlighter
[info] Exited with code 0

> -- Netbeans: very tricky with rc1 on Mac. If you do not succeed, I know
> how it works.
>
> -- building from source on Mac (especially 64 Bit): a few tweaks
> necessary
>
> Both things I will finally document for rc2 next week. If you need help
> before: mail to: https://launchpad.net/~raimund-hocke.
>

Thanks.

>
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Regards,
- Yarko

On Tue, Feb 8, 2011 at 12:56 PM, Yarko Tymciurak <email address hidden> wrote:

> On Tue, Feb 8, 2011 at 12:22 PM, RaiMan <
> <email address hidden>> wrote:
>
>> Your question #144310 on Sikuli changed:
>> https://answers.launchpad.net/sikuli/+question/144310
>>
>> RaiMan posted a new comment:
>> last try (just downloaded rc1, tried and this works):
>>
>> b = App("Firefox")
>> b.focus()
>> while not b.window():
>> wait(1)
>> Region(b.window()).highlight(2)
>>
>

in fact, this works too, so there is something different between setting
focus on your and my systems:
----
b = App("Firefox3")
b.focus()
while not b.window():
  wait(1)
# either of the following two lines work:
Region(b.window()).highlight(0.1) # this makes a difference;
# b.focus() # this causes reminder to work too...
type("l", KEY_META); wait(0.5)
paste("www.sikuli.org"); type(Key.ENTER)

--
Regards,
- Yarko

but if this works, then this should do too:

b = App("Firefox3")
b.focus()
while not b.window():
  wait(1)
type("l", KEY_META); wait(0.5)
paste("www.sikuli.org"); type(Key.ENTER)

why:
- Region.highlight() has no "connection" to the underlying region it is only a red frame in an "passive" overlay window.
- the second b.focus() should not be necessary, since b has already the focus

The only thing I could imagine would be to put another wait(1) before the type.

On Tue, Feb 8, 2011 at 1:24 PM, RaiMan <<email address hidden>
> wrote:

> Your question #144310 on Sikuli changed:
> https://answers.launchpad.net/sikuli/+question/144310
>
> RaiMan posted a new comment:
> but if this works, then this should do too:
>
> b = App("Firefox3")
> b.focus()
> while not b.window():
> wait(1)
> type("l", KEY_META); wait(0.5)
> paste("www.sikuli.org"); type(Key.ENTER)
>
> why:
> - Region.highlight() has no "connection" to the underlying region it is
> only a red frame in an "passive" overlay window.
> - the second b.focus() should not be necessary, since b has already the
> focus
>

You are right .... this does work...

I looked at my original post (about while loop failing), and I tried
renaming this (above - once it worked) from b.xxx to browser.xxx to see if
the variable name (accidentally) had some impact.

Nope - that works too (today / now).

Best I can guess: either there is something inconsistent (and I wish I
could have caught what it was in a debugger when it showed) ---- or there is
some interaction with something else in the system (I have McAffee virus
protection running, as a work requirement).

Also - as you can guess (it's os-x after all) - I don't reboot often
(monthly?). After this failed, I did reboot and still saw failure (I am
pretty sure I did that before my initial post).

Thanks for all your attention.

>
> The only thing I could imagine would be to put another wait(1) before
> the type.
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Regards,
- Yarko

Always welcome.

Hope you make good experiences with Sikuli.

pls. set question to solved.

So - I am able to do this:

I can reproduce, if I use original post.
Once reproduced, even things that "worked" in this list fail.
I spent some time narrowing it down, and finally ...
... ran out of steam for today - but I note:

b = App(someapp)
while not b.window():
     # something, something;
     b.focus()
     wait(1)

seems to have some problems, where:

b=App(someapp)
b.focus()
whille not b.windown():
    wait(1)

seems to be ok....

I did not really have enough time to narrow down to smallest consistently reproducible set; maybe I'll have a little more time tomorrow.