How to properly use observe to track region changes

Asked by Tristan Zickovich on 2017-10-26

This question pertains to the SikuliX 1.1.1 Java API.

I am trying to figure out how to track a region I have defined.
Specifically, my region encapsulates a character on the screen. I want to observe when he stops moving.
When he stops moving, I want to call a different function.

I believe I should use observe() and track onChange(), which will tell me when the character is moving. And when
onChange() reports false, it will tell me my character has stopped.

But I cannot seem to figure out how to get the onChange results()
to evaluate when he's moving and when he's not. I am not setting up the
observe functionality correctly.

This is what I have so far:

public void targetChanged(SikuliEvent e){
 Region x = e.getRegion();
 System.out.println(x);
}
public static void main(String[] args){
        Screen s = new Screen();
 Region r = new Region(s.getBounds());
 Location l = r.getCenter();
 Region characterRegion = new Region(l.x, l.y, 30, 40).offset(new Location(-30/2, -40/2));
 characterRegion.onChange();
 characterRegion.observe(1);
}

Currently the program just terminates.
What do I need to change to get my desired functionality?
I'm having trouble figuring out the documentation.
Detailed advice, explanations, and examples are appreciated.

Question information

Language:
English Edit question
Status:
Solved
For:
Sikuli Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
2017-10-28
Last query:
2017-10-28
Last reply:
2017-10-27
RaiMan (raimund-hocke) said : #1

ok, I have to admit, that for Java the docs are not really helpful - I will optimize that ;-)

Here is a working example for an onChange observation:

a helper for the formatted printout:
  private static void p(String msg, Object... args) {
    System.out.println(String.format(msg, args));
  }

the example:
      Region reg = new Region(0,0,80,80); // define the observed region
      reg.highlight(2);
      reg.onChange(new ObserverCallBack(){ // define the handler
        @Override
        public void changed(ObserveEvent evt) {
          if (evt.getCount() > 3) {
            p("in handler: %d - stopping", evt.getCount());
            evt.stopObserver();
          } else {
            p("in handler: %d", evt.getCount());
          }
        }
      });
      reg.observeInBackground(); // start observation in background forever
      int n = 0;
      while (reg.isObserving()) { // do something while observe is running
        p("%d - observing", n);
        reg.wait(1.0);
        n++;
      }

which prints something like this:
[log] highlight R[0,0 80x80]@S(0) for 2.0 secs
0 - observing
in handler: 1
1 - observing
2 - observing
3 - observing
4 - observing
in handler: 2
5 - observing
in handler: 3
6 - observing
in handler: 4 - stopping

--- some comments
- if you want to track some motion or some continuous change, then you have to either loop around an observe(time) or use a construct like the above example with an observe in background.
- in your main workflow, you have to track wether anything happens at all and if it does, when nothing happens anymore (which is not implemented in the example), but which is needed in your case.

come back if you need more help

RaiMan (raimund-hocke) said : #2

BTW: you have to use the 1.1.2 nightly build in about 2 hours

With the code you provided, reg.isObserving() always evaluates to false on my end.
However, without the while loop, the event handler works fine.

Also, observeInBackground() requires a parameter of type double. The call without a parameter does not work for me.
Is there another way to run it forever, other than declaring a huge double value?

Is there a way to tell the % difference between changes? I've seen other Sikuli functions return a similarity parameter.
I am calling :
         characterRegion.onChange(new ObserverCallBack(){ // define the handler
  @Override
  public void changed(ObserveEvent evt) {
   List<Match> characterChanges = evt.getChanges();
   for(int i = 0; i < characterChanges.size();++i) {
    System.out.println(characterChanges.get(i));
   }
  }
 });

evt.getChanges() is the only function for ObserveEvent that I've tried that doesn't return null.

This is a small sample set of my moving character:
M[956,521 8x18]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,530 [-1 msec]
M[956,531 8x8]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,535 [-1 msec]
M[956,529 8x10]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,534 [-1 msec]
M[956,521 8x18]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,530 [-1 msec]

And this of my "still" character:
M[956,521 8x9]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,525 [-1 msec]
M[956,521 8x8]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,525 [-1 msec]
M[956,521 8x7]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,524 [-1 msec]
M[956,522 8x8]@S(S(0)[0,0 1920x1080]) S:-1.00 C:960,526 [-1 msec]

Even my still character slightly shifts in place. So I need to find a way to differentiate.
The only thing i see is the H of M, and the second variable of C.
But is there a way to find how much of the image actually changed during the event handler call?

Also, where do I find this 1.1.2 nightly build? the SikuliX page still says 1.1.1 is the current version.

Best RaiMan (raimund-hocke) said : #4

--- Also, where do I find this 1.1.2 nightly build?
http://sikulix.com -> Actions bar -> Get nightly builds

As mentioned for my example:
To use reg.isObserving() properly, you need 1.1.2 nightly. Same goes for observeInBackground().

The M[] values denote the region, that contains the changes (smallest area that contains all changed pixels).
The C value is only the center of M[] and hence redundant.

In fact I cannot see, that your character moves in the sense, that its location on the screen changes.
The left side and the width are always the same, it only changes its height:
int actualHeight = characterChanges.get(i).h

--- But is there a way to find how much of the image actually changed
you might play with the changed pixel parameter of onChange()

Thanks RaiMan, that solved my question.