SikuliX

How to capture mouse cursor --- not possible in the moment

Asked by Jason

Is it possible to capture mouse cursor with Sikuli? I want to check if my mouse cursor icon changes when I do some actions.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Best RaiMan (raimund-hocke) said : 		#1

- the capture methods (from IDE or hotkey) need the mouse cursor to make the rectangle - no chance to capture the cursor itself.

- using an capture(region) in a script that should contain the mouse cursor, you find, it is not visible. Wether the screenshot that is made to find the visual object contains the cursor at all: I do not know, since these screenshots are no longer saved to disk and kept in memory.

- on my Mac I can change the cursor size to very big. then they seem to use another technique to show the cursor and it gets visible on an internal capture and is captured and must be on the screenshot, since it influences the match (tested)

- and even, if you find a way to get an image of your cursor, it is always a rectangle, that contains some background at the time it is photographed. So it might be hard to match it on other backgrounds.

Conclusion: little hope to find a solution on the Sikuli script level. May be there is a chance on the Java level with the awt robot.

Revision history for this message
Jason (jasonyoo) said : 		#2

Thanks RaiMan, that solved my question.

Revision history for this message
Chris (izanagi-xiii) said : 		#3

Just a thought don't know if its possible will have a go when I get some time tonight.

What you could perhaps do is change your mouse pointer to a different icon which you create. This would mean that the mouse pointer would be unique and you can take the image of it have the image of it.

If you make it a square this would be ideal for capture as RaiMan said as it wouldn't contain a background which was out of your control.

Otherwise I can only recommend something like SnagIt (image capture software) which has the option to include the pointer in its screenshots. This would work if you know where the pointer will be. - if you really wanted to you could then screenshot this image using the Sikuli IDE and it would be in there - don't know if this would degrade the image though.

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

@ Chris
good ideas.

You make a follow up for Windows, I do it for Mac - ok?

Comments:
-- yes: try to make the cursor a square or different rectangles if you need to change the cursor (as the primary request was)

-- yes: use other screenshot tool if you do not succeed with Sikuli's features
--- pointer location can be evaluated by Env.getMouseLocation()
--- external SCTool can be some how triggered from within Sikuli using e.g. keyboard shortcuts
--- the saved image should be in .png format, then you could use it directly in Sikuli using a Finder() on that screenshot to check for your cursor

If you come back with a solution, I can help you to make it a FAQ.

Revision history for this message
Chris (izanagi-xiii) said : 		#5

I gave this a quick test from the view of getting the mouse to change to the custom icon - it works you just need to know what action you are doing i.e

Normal Select
Help Select
Working in Background
Busy
etc....

then apply the image to that mouse action in the Mouse properties.

NOTE: I used an icon converter program to convert the image I made to a .ico .cur will also work. Keep the PNG for use in Sikuli.

Quick Test:
I set my Normal Select action to show my image rather than the normal arrow then when moving over something where another action can occur i.e Window Expand type icon, my custom icon disappeared.

I haven't checked in Sikuli yet but I don't see a reason why this couldn't be tracked. Perhaps by observe or if expected after a certain action by an if statement.

Will try and check this tomorrow in Sikuli itself. (I need to install sikuli on my W7-64bit for home use don't have the time to edit the .bat file tonight.)

Revision history for this message
RaiMan (raimund-hocke) said : 		#6

Uuups, Win7 64-Bit? Might be that Sikuli will not work.

see: bug 702797 (if you run into problems, add yourself to the "I am affected" queue. If you can solve it somehow, put a comment on the bug, about the things you have done)

In case of problems: If you tell me how to produce .ico or .cur files on Win7, I can test the Sikuli part on my Win7-32.