SikuliX

UTF-8 & type(), any other ways?

Asked by DenisD

Hi guys, RaiMan!
First of all I'd like to thank you for the new life for Sikuli! I've admired this project.

Today I found that UTF-8 characters aren't supported by type(). I got following error in my java tests:

Exception in thread "main" java.lang.IllegalArgumentException: Cannot convert character Д
 at org.sikuli.script.Key.toJavaKeyCode(Key.java:319)
 at org.sikuli.script.RobotDesktop.typeChar(RobotDesktop.java:234)
 at org.sikuli.script.Region.keyin(Region.java:2748)
 at org.sikuli.script.Region.type(Region.java:2613)
 at kz.vas.emulator.whatsapp.WhatsAppSender.main(WhatsAppSender.java:22)

I found many questions on the portal concerning this situation and understand that for type() I must use ASCII or use paste() to past UTF-8 symbols. However, I'm testing android app on the emulator and there is no option to use past(), and type() is the only option to input text (as I may suppose).

So, my question is: is there any opportunities to input text using sikuli to android apps? Perhaps, you know additional resources which I can include to my project?

RaiMan, guys, could you please advise me whether you have any instruction about source codes? Due to the fact that there are many packages I'm a bit confused, and couldn't find source codes of API for java (I mean source codes for sikuli-java.jar). I'd like to browse the code, perhaps I could fix the UTF problem.

Thank you!

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
RaiMan
Solved:
Last query:
Last reply:
Revision history for this message
Best RaiMan (raimund-hocke) said : 		#1

The Java AWT Robot function used down the road for Sikuli's type() on the Java level is restricted to the use of Java defined key constants, that are mapped to the standard layout of the US-EN qwerty keyboard.

So the usage of the type() generally is restricted to whatever you can produce with your keyboard (which in fact includes some on-ascii characters on many non-EN keyboards). The current version of Sikuli up to 1.0.1 is restricted, to a subset of what Java finally "presses" on your keyboard, taking the given character as the one from the US-layout.
Some restrictions can be overcome, if you know, which US-EN-key to press, to produce a wanted character with your local keyboard layout.

With version 1.1.0 there will be a tool, that helps to generate a translation table for your used keyboard and after activation you can use the characters as they are defined for your keyboard and they will be typed as such.

But what is really missing is an approach, to generate "any" utf-8 character with the keyboard interface of Java AWT Robot.
... but currently, I have no idea how to do that and what tool/library to use for that.
... and this of course is a different solution on the different systems Windows/Mac/Unix --- grrrrr ... I hate it ;-)

On Windows, I know, there is a trick with the num keys and the alt/alt-gr key to produce non-ascii characters:
press and hold alt and press the 3-digit decimal representation of the character code on the num pad.
If this is a solution for you: you can do that with type - you only need a translation function.

The actual coding of Sikuli is on github (https://github.com/RaiMan/SikuliX-2014).
The key translation is done in the class Key.
The interface to Java AWT Robot is in class DesktopRobot.

If you find A-N-Y solution or have any idea how to get nearer: your contribution is highly appreciated.

Revision history for this message
RaiMan (raimund-hocke) said : 		#2

Hey, just found that:
http://www.fileformat.info/tip/microsoft/enter_unicode.htm

... I will have a look, how to implement it into the Key class.

Revision history for this message
DenisD (denis-work-acc) said : 		#3

RaiMan, thank you very much! Now it's clear for me why such situation happens.
Yes, it's logical. How would have I supposed to type UTF symbols if I hadn't even changed the layout :)

It seems to me that me neither have solution for the AWT robot at the moment... sorry... But what I can do is I can provide the "mapping tables" for Russian and Kazakh languages in any convenient for you type, if it's helpful of course. And I'll definitely see Key class, m.b. my mind would be enough to make changes there.

Thank you again!

Revision history for this message
DenisD (denis-work-acc) said : 		#4

Thanks RaiMan, that solved my question.

Revision history for this message
RaiMan (raimund-hocke) said : 		#5

Please wait until I publish the 1.1.0-Beta1 in second half of April, then you can produce the translation tables for different keyboard setups out of the box.

thank you.

Revision history for this message
DenisD (denis-work-acc) said : 		#6

Yes, of course :) I'll be waiting the update

Revision history for this message
NoBugs! (luke32j) said : 		#7

So are special characters supported in 1.1 nightly? I have a recent 1.1 nightly build and I was getting fatal exception when typing something with ñ character.

Revision history for this message
RaiMan (raimund-hocke) said : 		#8

as the related bug tells you:
planned for version 2 only.