How to Copy text from a regions?

Asked by Joe Harris

I want sikuli to copy the text from regions and paste it in some other window. The text will be dynamic, it will be generated different text strings after pressing a button. Can Sikuli Copy and paste it to some other window?

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Joe Harris (b5360657) said :
#1

Update: Or I say BUMP? Please help?
I need to copy some text which will be generated by some website in a region

Revision history for this message
Willie (willie-owens91) said :
#2

Create a new Region where the text should appear, I believe you can do so with exact or relative coordinates. Then use the text() method to extract the text from that region (it is returned as a string). Be aware the the text recognition is still in an experimental state.

Revision history for this message
RaiMan (raimund-hocke) said :
#3

If Willie's suggestion (which is the primary solution to try) does not satisfy you and the text to copy is selectable with mouse and/or keyboard you might simulate either

-- a copy/paste action (somehow-select-the-text/ctrl-c/click(target-region)/ctrl-v) or

-- a drag/drop-operation (somehow-select-the-text/mouseDown()/mouseMove(target-region)/mouseUp()).

If the text is somehow selectable, this method works in any cases - but it needs some more coding and you might have to solve timing problems.
(an example: http://sikuli.org/blog/2010/05/06/extract-text-with-clipboard/)

Revision history for this message
Joe Harris (b5360657) said :
#4

I tried I cannot do that, it always give me error or copy whole stuff from the screen
this is code which gives me error
text("region from where to copy text")
paste(text,"region to paste text" )
. Please help?

Revision history for this message
RaiMan (raimund-hocke) said :
#5

the usage is:
theText = find("some-image").text()
paste("region to paste text", theText)

pls. look into docs: http://sikuli.org/docx/

Revision history for this message
Joe Harris (b5360657) said :
#6

But I want to extract some Variable string. I cannot use the find function to grab it, because It will result findfailed, If i put the image of the text. I want to copy from a region to region

For example :-
I want Sikuli to copy heading of this question page(https://answers.launchpad.net/sikuli/+question/162592)
If we assume that user is scrolled top of the page

Sikuli should copy from Region(20,168,447,38)

Revision history for this message
RaiMan (raimund-hocke) said :
#7

maximize this question page and scroll to top and make it the frontmost

try this script as is

dir = "http://its-me-raiman.de/sikuli/images/"
sik = dir+"sikuli-logo-lp.png"
m=find(sik).below(42).left(1).right(600)
m.highlight(3)
print m.text()

if it works, there should be in the message area afterwards:
How to Copy text from a regions

Hope it helps to understand

the sikuli logo image is loaded from the web at
http://its-me-raiman.de/sikuli/images/sikuli-logo-lp.png

If you get a FindFailed, you have to capture the logo on your machine and replace in find(sik) sik with your capture.

Revision history for this message
Joe Harris (b5360657) said :
#8

Thank you :-) Okay, it worked, But It doesn't Highlight the whole "HOW" instead the it starts highlights just the -|, It's hard to explain, Here's a screenshot :- http://k.min.us/idDlk8.bmp

I tried changing the value of Left() to 0.5 and even -100, but it won't move it.
Also how did you get those values of below() left() right()?

Revision history for this message
RaiMan (raimund-hocke) said :
#9

OK, this is some rendering difference between our 2 browsers (mine: Safari on Mac).

you might easily adjust this by adding a correction for the x coordinate of m:

m=find(sik).below(42).left(1).right(600)
m.x = m.x - 10
m.highlight(3)

or you might insert another left():
m=find(sik).below(42).left(10).left(1).right(600)

--- I tried changing the value of Left() to 0.5 and even -100, but it won't move it.
Sorry, the left(1) is a hack. same goes for the use of above(1) ;-)

the goal is to get a region of hight 42 and a width of 600 below the match of the sikuli logo.

you might do it this way:

s = find(sik)
m = Region(m.x-10, m.y+m.h, 600, 42)

I prefer the above notation, since it is more readable.

m=find(sik).below(42).left(10).left(1).right(600)

in words: find the sikuli logo and then make a region below the sikuli logo with height 42 and then make a region left of this with the same height and width 10 (this adjusts the left border) and then make a region left of this with the same height and width 1 (I take a 1 as width to "show" it is the hack, this finally positions the left border reference) and then make a region right of this with the same height and a width of 600.

--- Also how did you get those values of below() left() right()?
Try and error or a screen shot tool, that shows coordinates.
see faq 1686

Revision history for this message
RaiMan (raimund-hocke) said :
#10

Sorry, correction:

you might do it this way:

s = find(sik)
m = Region(s.x-10, s.y+s.h, 600, 42)

Revision history for this message
Joe Harris (b5360657) said :
#11

Thank you ! :)
It worked, but Sikuli's Ocr read it as "Howto Copy text from a regions?", and tried on many others too.
Always read the incorrect :(

Isn't there any copy command in sikuli?

By which we can simply copy the Text in the region, I think maybe somehow highlight with mouse click, move to right, key.type(Ctrl, "v"). be the only way to get the same text?

So I want to
Double click on the region (So it highlights) > Press Shift key then press Down key to select the whole line > Press Ctrl v to copy.

Code :
doubleClick(Region(397,524,14,18))
type(Key.SHIFT+Key.DOWN)

it's doesn't highlights the whole line. I see
"[log] DOUBLE CLICK on (404,533)
[log] TYPE ""

In the message bar for the following. :(

Revision history for this message
Best RaiMan (raimund-hocke) said :
#12

--- Sikuli's OCR
If you want a reliable text copy feature: it is not usable currently.

--- type(Key.SHIFT+Key.DOWN)
this will never work, since in the combination shift-down, the shift is not a single key press, it is a modifier key, that goes together with the down key.

for these key combos you have to write:
type(Key.DOWN, KEY_SHIFT)

--- selecting text with mouse/keyboard actions
that is principally possible, but depends on the application.
In web pages, you usually have to select with the mouse.
In editor apps or editable sections in web pages clicking and then use keyboard actions usually works.

The challenge always is, to find e.g. the start point for the mouse selection and to know where to move the mouse.

So again an example for this page:

Suppose, we have a pixel location, that is left of the first character of the headline, then press and hold left button and move the mouse to the right side until right of the last character, release the mouse and fire a ctrl-v.

dir = "http://its-me-raiman.de/sikuli/images/"
sik = dir+"sikuli-logo-lp.png"
m=find(sik).below(42).left(1).right(600)
m.highlight(3)
leftPoint = m.getTopLeft().below(m.h/2)
rightPoint = m.getTopRight().below(m.h/2)
click(leftPoint)
keyDown(Key.SHIFT)
mouseDown(Button.LEFT)
hover(rightPoint)
mouseUp()
keyUp()
type("c", KEY_CTRL)
click(leftPoint) # to reset the selection
wait(3) # only to have a chance to see what happens
print Env.getClipboard()

Revision history for this message
Joe Harris (b5360657) said :
#13

Great That's Awesome Script. (must've took alot of time to write) :D
It fine, except of the region was not right, maybe because I'm taking screenshot of the sikuli logo, As I cannot use the one from your website.
I got idea to it, Thanks alot for your time, RaiMan :))

Revision history for this message
Joe Harris (b5360657) said :
#14

Thanks RaiMan, that solved my question.