OpenCV Error on starting event observer

Asked by Terence Cooper

This happens when I have registered, triggered and stopped an onChange event observer several times. (I'm using the updated sikuli-script.jar you (RaiMan) sent me last week (I'm using the java api)).

[RaiMan] observer started
[log] CLICK on (227,10)
targetChanged called
stopping observer
stopped observer
[log] highlight Match[186,23 257x215 score=-1.00 target=center] for 1.0 secs
[RaiMan] observer started
targetChanged called
stopping observer
stopped observer
[log] highlight Match[195,24 239x24 score=-1.00 target=center] for 1.0 secs
[log] highlight Match[205,4 44x12 score=1.00 target=center] for 1.0 secs
[log] CLICK on (227,10)
about to register observer
[log] highlight Match[186,48 239x191 score=-1.00 target=center] for 1.0 secs
registered observer
[RaiMan] observer started
OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in Mat, file /Users/vgod/Downloads/OpenCV-2.2.0/modules/core/src/matrix.cpp, line 307
terminate called after throwing an instance of 'cv::Exception'
  what(): /Users/vgod/Downloads/OpenCV-2.2.0/modules/core/src/matrix.cpp:307: error: (-215) 0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows in function Mat

Process finished with exit code 134

Immediately after registering the onChange event observer, I do a region.mousemove and then sleep. When I run the script, It looks as though the mouse starts to move, then I get the OpenCV error.

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
RaiMan Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said :
#1

I will look at it tomorrow.

pls. send me the relevant code snippet and a screenshot.

Revision history for this message
Terence Cooper (terence-o) said :
#2

Also, targetChanged is returning the same match for different changes in the observed region - even after stopping observations and calling region.onChange again.

Revision history for this message
Terence Cooper (terence-o) said :
#3

ok, I'll have to clean up a bit!

Revision history for this message
Terence Cooper (terence-o) said :
#4

The openCV error message appears with the 2nd call to mapSubMenu. Also the changed area found in the 1st call to mapSubMenu is highlighted again when the 2nd call is made, ie. targetChanged is returning the same match for different changes in the observed region. When I run this, I first see the TxDev menu area highlighted, then the pointer moves to click on the *Search* menu, and then just as it begins to move again the the stack is dumped.

 Plz don't hesitate to let me know if you need more info. Thx in advance. Terry.

    static void mapSubMenu(String mainMenuLabel, String[] subMenuLabels)
    throws FindFailed, InterruptedException {

        monitoredR = subMenusR;
        changedM = null;
        getChangedAreaMatch();
        // click the menu...
        mainMenu.click(mainMenuLabel);
        //mainThread.wait();
        Thread.currentThread().sleep(2000);
        Region menuR = changedM;
        menuR.highlight(1);

        try{
            double ms =  Settings.MinSimilarity;
            Settings.MinSimilarity = .4;
            for(String  menuLabel: subMenuLabels){
                menuR.highlight(1);
                Match menuItemM = menuR.wait(menuLabel, 30);
                menuR.h = menuR.h+menuR.y - (menuItemM.y+menuItemM.h);
                menuR.y = menuItemM.y + menuItemM.h;
                menuItemM.highlight(1);
                MenuItem subMenuItem = new MenuItem(menuLabel, "");
                subMenuItem.setMatch(menuItemM);
                subMenuItem.setParent(mainMenu);
                subMenuItem.setHighlighted(false);
                subMenuItem.setOnDisplay(false);
                mainMenu.getChild(mainMenuLabel).addChild(menuLabel, subMenuItem);
            }
            Settings.MinSimilarity = ms;
        }
        catch(FindFailed findFailedExcptn){
            scrpt.popup("Failed to map "+mainMenuLabel+" menu\n - "+findFailedExcptn.getMessage());
            exit(0);
        }
        mainMenu.click(mainMenuLabel);                   // close menu

    }

    static void getChangedAreaMatch()
    throws FindFailed{

        monitoredR.onChange(50, new SikuliEventObserver(){
            public void targetChanged(ChangeEvent evnt) {
                System.out.println("targetChanged called");

                for (Match m : evnt.changes) {
                    changedM = m;
                    m.highlight(1);
                    break;
                }
                System.out.println("stopping observer");
                evnt.region.stopObserver();
                System.out.println("stopped observer");
                //monitoredR.stopObserver(); // which one?

            }

            public void targetAppeared(AppearEvent e) {
                scrpt.popup("targetAppeared called");
            }

            public void targetVanished(VanishEvent e) {
                scrpt.popup("targetVanished called");
            }
        });

        // start the observation...
        monitoredR.observeInBackground(30);

    }

    public static void main(String[] args)
            throws InterruptedException,
            java.awt.AWTException,
            org.sikuli.script.FindFailed{

        scrpt = new SikuliScript();
        scrn = new Screen();

BLAH, BLAH, BLAH

        mapMainMenu(); // (doesn't use the observer and onChange event handler)

        mapSubMenu("TXDev" ,txDevMenuLabels);

        mapSubMenu("Search", searchMenuLabels);

BLAH, BLAH, BLAH

    }

}

Revision history for this message
Terence Cooper (terence-o) said :
#5

Looks like 1. stopObserver isn't stopping the observation immediately (maybe the observeInBackground timeout is being respected despite the call to stopObserver) and 2. subsequent calls to targetChanged after reregistering a SikuliEventObserver are returning matches made in the previous (extended) observation period. (This may be pure fantasy!!). Terry.

Revision history for this message
Terence Cooper (terence-o) said :
#6

I think I have partially verified my theory (last comment). To recap... I am monitoring a region in which menus appear when a menu item in the menu bar is clicked. I am stopping the observation in the change event handler as soon as the menu appears, or at least that's what I'm trying to do. Having got a match for the area where the menu appears, I then search that region for the menu items on that menu (using text recognition). I then close the menu, start the observation again and open the next menu from the menu bar. At this point I am getting spurious changed area matches in the area of the 1st menu which appear to be the result of closing the 1st menu and these change matches are for changes which have occurred *after* the observation should have ended.

In order to verify my suspicions (previous comment) I now determine which thread the observer is running in, and stop that thread in the change event handler. This reduces the the number of matches to 1 - presumably the change is the appearance of the 2nd menu, however the changed match area corresponds with the area of the 1st menu, not the 2nd...

Revision history for this message
RaiMan (raimund-hocke) said :
#7

This refers to comment #4:

Your approach principally does not really make sense.

The value in changeM is an arbitrary region of 50 pixels (about 7x7), that has changed in the observed region (monitoredR = subMenusR --- BTW: why not a parameter to mapSubMenu). Normally it is not useable for anything.

The area, that you have to look for your submenu texts is the space below the match returned by mainMenu.click(mainMenuLabel);

The "changed regions" return value does not contain the region in total that has changed. You might analyze the returned changes with a strategy "most-top-left-change" and "most-bottom-right change" and construct the whole "change region" from that.

BTW: dealing with the observe subprocesses your self is at least risky, since you might leave the Sikuli environment in an undefined state. I doubt the developers have not programmed anything for such a situation.

Revision history for this message
Terence Cooper (terence-o) said :
#8

Hi RaiMan, thanks for your response. I think I have failed to explain my problem adequately.
I can see the code in the snippet I provided is misleading in some ways. I'm certain there is
something amiss here, so I'll create a simple complete stand-alone example to demo the
problem. I've used the top-left bottom-right matching strategy elsewhere - I was trying
to use an approach that didn't depend on image matching. As for messing with the Sikuli
threads, I did that only to try to debug the problem and certainly wouldn't consider
doing such a thing in real code. It'll be a few days before I can provide you the example I
mentioned. I have to make progress, so I'll revert to the old matching strategy for now.
Thanks much for your efforts. Terry.

Revision history for this message
RaiMan (raimund-hocke) said :
#9

Thanks for feedback. I will meanwhile try to check wether my approach would work.
All the best.

Can you help with this problem?

Provide an answer of your own, or ask Terence Cooper for more information if necessary.

To post a message you must log in.