How to make sikuli detect motions within a specific region?
I'm just wondering if sikuli has the ability to detect motions on the screen, e.g. I have a video playing on the screen and detect the pixel is changing within the player window?
video = App('vlc.
I know sikuli has some observation functionality, but I'm just wondering how I approach this problem?
Some lines of codes might be helpful...
Thanks!
Question information
- Language:
- English Edit question
- Status:
- Answered
- For:
- Sikuli Edit question
- Assignee:
- No assignee Edit question
- Last query:
- 2013-05-06
- Last reply:
- 2013-06-12
RaiMan (raimund-hocke) said : | #1 |
the simplest method:
reg = some_region_
img = capture(reg) # take a shot
wait(1)
while reg.exists(
print "region content changed
for usage of observe look http://
Frank Feng (ffeng) said : | #2 |
Thanks, RaiMan.
I was trying to get through some sample code. I need to call sikuli apis on Jython level. Here are some code:
from sikuli import Sikuli
def changed(event):
event = Sikuli.SikuliEvent
print "Something changed in ", event.region
print event.changes
print dir(event.changes)
print help(event.changes)
for ch in event.changes:
Sikuli.wait(2)
for ch in event.changes:
with Sikuli.
Sikuli.
Sikuli.
Sikuli.wait(10)
r.stopObserver()
Then I run jython script.py, and tried to select a video playback then it gave me this error:
None
Exception in thread "Thread-5" Traceback (most recent call last):
File "C:\Automation_
File "C:\Automation_
File "Sikuli_app.py", line 17, in changed
for ch in event.changes:
TypeError: 'reflectedfield' object is not iterable
Can you please help?
RaiMan (raimund-hocke) said : | #3 |
You overwrite the handler parameter event, which is set outside, so it gets invalid.
this is a version using inline/foreground observation (starting the observer, stop it in the handler, restart it in main):
from sikuli import *
def changed(event):
# event = Sikuli.SikuliEvent
print "Something changed in ", event.region
print event.changes
# print dir(event.changes) # changes has no methods
# print help(event.changes) ???
for ch in event.changes:
# makes no sense, because changes are not updated while in handler
# Sikuli.wait(2)
# for ch in event.changes:
# ch.highlight()
event.
r = selectRegion(
r.onChange(50, changed)
for i in range 10: # or some timed loop
r.observe(
wait(2)
This is a suitable version, if some processing is done in the handler(the observation is paused, when in the handler!)
This version uses the background observation and needs some communication between handler and main:
from sikuli import *
def changed(event):
print "Something changed in ", event.region
print event.changes
# content of highlighted areas might have changed already!
for ch in event.changes:
wait(2) # pauses observation
Settings.
r = selectRegion(
r.onChange(50, changed)
r.observe(FOREVER, True)
Settings.isChanged = False # "pseudo" global, some primitive handler communication
for i in range 10:
if r.isChanged:
print "something changed"
wait(2)
r.stopObserver()
BTW: wether this works to your satisfaction heavily depends on, how often per second the content in the video area changes. It might be hard to really get the changes you want. But give it a try.
Frank Feng (ffeng) said : | #4 |
Hi RaiMan,
Thanks for the help. What I'm trying to do it open an video using a certain player and then keep checking if the pixels within that region kept changing , then I can assume the video was playing. So I'm wondering if the onChange function can return a status of the current region it's observing, if nothing changed at all, then I know something went wrong, say I observe for a certain period of time, and I'm pretty sure that the video is much longer than that. If the onChange function gave me a status saying 'nothing changed', then I can raise an exception from there. I noticed the event.changes will return a match object, so Can I check if this object is None, then I stop observing and throw an exception. Here are some codes that I can think of:
def changed(event):
print "Something changed in ", event.region
print event.changes
for ch in event.changes:
if ch is None:
raise Exception('the video stops playing')
def main():
r = GetAppRegion(
print r
if r is None:
raise Exception('No App region captured')#just double check
r.onChange(
while True:
if r.exists(
print "end of video"
break
print "******
print "test done!!!
print "******
Thanks again!
Frank Feng (ffeng) said : | #5 |
sorry, i made a mistake, actually the following line is not right:
for ch in event.changes:
if ch is None:
raise Exception('the video stops playing')
'ch' will never get to 'None' i guess...
Frank Feng (ffeng) said : | #6 |
Again, I mean if there is a way to check if the region kept changing or not....
Sorry for being so wordy...
Thank.
RaiMan (raimund-hocke) said : | #7 |
If you want to detect, wether the video stops at a time it should not do that, then looking for changed pixels with the foreground observer is not the right way, because your script will simply hang at the observer(), when the video stops.
Hence you have to use the background observe() (see version 2 in comment #3), to be able to check, that nothing changed within the last x seconds.
But for your purpose, the following might be sufficient:
r = GetAppRegion(
img = capture(r)
wait(1)
while not r.exists(
wait(1)
img = capture(r)
wait(1)
print "video seems to have stopped"
this takes an image of the region, waits 1 second and then checks, wether the region content is still the same - if not, waits 1 second and starts all over.
Aravind (a4aravind) said : | #8 |
Hi Raiman,
Sorry to reopen this thread but I have a small doubt regarding the "r.isChanged" in Comment#3.
The same might have turned out to be really helpful for report generation if that r.isChanged worked fine.
Now while running the script, it says
AttributeError: 'Region' object has no attribute 'isChanged'
Here goes the code snippet which I've tried based on your logic:
bigScreenReg = Region(
bigScreenReg.
print "Checking bigScreen Video Region"
r = bigScreenReg
r.onChange(100, changed)
r.observe(
Settings.isChanged = False
for i in range(5):
if r.isChanged:
print "Something changed"
else:
print "Nothing changed"
r.stopObserver()
And the following is the handler:
def changed(event):
print "something changed in ", event.region
for ch in event.changes:
sleep(1)
for ch in event.changes:
Settings.
Or is there any other work around to get the 'Video stopped streaming' message in the report by sticking on with the onChange event handler itself?
Thanks a lot Raiman.
RaiMan (raimund-hocke) said : | #9 |
Uuups, I did some crap in the above mentioned code - sorry.
this should do it:
Here goes the code snippet which I've tried based on your logic: (modified ;-)
bigScreenReg = Region(
bigScreenReg.
print "Checking bigScreen Video Region"
r = bigScreenReg
r.onChange(100, changed)
Settings.isChanged = False
r.observe(5) # observing for 5 seconds (script pauses here)
if Settings.isChanged: # check wether the handler was called (because there where changes)
print "there where changes"
else:
print "there haven't been any changes"
And the following is the handler:
def changed(event):
print "something changed in ", event.region
for ch in event.changes:
sleep(1)
for ch in event.changes:
Settings.
event.
--- comment
You are using the foreground observe in this case, the you should stop the observation in the handler.
hope it helps
Aravind (a4aravind) said : | #10 |
Thanks very much Raiman...That solved my problem :)
xyz_User (userseven) said : | #11 |
Hi RaiMan, I found this to be extremely helpful for my current project. Can I add a request to this: Once it detects changes, lets say it found 5 - 10 pixel changes: How can get sikuli to click only ONE of those changed pixels that the observer saw?
Can you help with this problem?
Provide an answer of your own, or ask Frank Feng for more information if necessary.