SikuliX

Drag or mouseDown won't grab a selection

Asked by Kyle VandenLangenberg

Hello,

So I am trying to dragDrop (drag, dropAt) a selection of two files from one open dialogue box in my program to another. I can left-mouse click and drag/drop these files myself (manually with my own mouse). However, when I use any of the functions I know dragDrop, drag - dropAt, and a combination of mouseDown(), mouseMove(), mouseUp(), its like Sikuli isn't actually pressing the button.

The mouse moves to the pattern where the drag starts, then moves to the pattern where the drop is to occur, but it doesn't "tow" the pattern along with it.

Any thoughts on this?

Thanks in advance.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

Might be a timing problem.

Try something like that:

mouseDown()
wait(0.5)
mouseMove()
wait(0.5)
mouseUp()

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#2

Actually tried that too, no luck.

When I manually do the dragDrop, once I mouseDown and start to move the mouse a graphic also appears that sort of indicates the drag is occuring (an example might be dragging a folder on your desktop, as you start to drag it on a mac, a transparent folder appears as you are moving the mouse).

 When Sikuli runs the script, no graphic appears. This suggests to me that the actuall mouseDown to start the drag isn't grabbing the object.

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

You are on a Mac?

Sure, the window has focus?

If not you must first click() to give focus.

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#4

The command immediately proceeding the drag() operation is a click() on the very same object.

Revision history for this message
RaiMan (raimund-hocke) said : 		#5

Windows or Mac?

What application?

checked longer wait times?

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#6

Mac OS 10.7.4

Application is DNASTAR SeqBuilder (bioinformatics software).

I tried wait times up to 1 second. I could try longer and report back.

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#7

Here is the section of code:

click("fileone.png")
click("filetwo.png", KeyModifier.SHIFT)
wait(1)
hover("bothfiles.png")
mouseDown(Button.LEFT)
wait(1)
mouseMove("dropfolder-1.png")
wait(1)
mouseUp(Button.LEFT)

Again, its like its not actually mousing-down.

Revision history for this message
Best RaiMan (raimund-hocke) said : 		#8

I tested on my Mac OSX 10.7.4
dragging a group of files from a Finder column to another column.

the following script worked as expected:

img = "QnativeFinde.png"
m = find(img).getCenter()
#hover(m)
click(m.below(4*18)) # click below to activate window
#select 4 files
click(m)
click(m.below(3*18), KeyModifier.SHIFT)
wait(2)
# drag
mouseDown(Button.LEFT)
wait(0.5)
mouseMove(m.left(50).below(50))
wait(0.5)
#drop
mouseUp(Button.LEFT)

But there is a strange behavior after having run this script:
The Finder window I acted on always stays in front, no matter where I click. Even after using the task switcher to come back to Sikuli-IDE, the next click brings back the Finder window. Closing the Finder window stops the behavior.

I do not know, what this means for your situation, but at least with Mac Lion there seems to be a problem with the usage of the Java Robot class mouse feature.

I have no idea for a workaround.

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#9

Thanks, I will get a chance to try this tomorrow and will see if it works.

One question, how does the coding for the 4*18 work? Is that pixels?

Thanks.

Revision history for this message
RaiMan (raimund-hocke) said : 		#10

my Finder column has 4 entries.
each entry line is 18 pixels high (measured using cmd-shift-4)

img = "QnativeFinde.png" # image of the first entry including part of the toolbar
m = find(img).getCenter() # pixel location (x,y) on the first entry
#hover(m)
click(m.below(4*18)) # click below to activate window

m.below(4*18)
creates a new pixel location by adding 72 to m's y value and hence will click in the white area just below the 4th entry and activate the Finder window.

I always do this with regular GUI situations while testing, so later I could replace variables:
nLines = 4
lineHeight = 18

Revision history for this message
Kyle VandenLangenberg (vandenk) said : 		#11

Thanks for the help, this actually didn't solve the problem - it seems like it may be an issue in my program.

I randomly tried something like this though and it worked:

hover(ImgIWantToMove)
mouseDown()
mouseMove(UpAFewPixels)
mosueMove(ToWhereIWantToMoveIt) #which is a different dialogue window in the same program
mouseUp()

this worked. Thanks.

Revision history for this message
RaiMan (raimund-hocke) said : 		#12

Interesting.

This means that the GUI does not accept this one fast mouse movement as a drag operation.

I just tested while writing this comment with the trackpad of my new MacBookAir:
- double click one word
- 3 finger drag slow: starts a drag operation
- 3 finger drag fast (this is the one-mousemove with Sikuli): selection is extended to end of line - no drag.

So: good catch ;-) and thanks for feedback.