SikuliX

How to click a specific UI element using regular expression

Asked by Siva

Hi,

I started using sikuli last week for UI Automation, need some help.

I am working on a windows mobile automation activity. Here is the scenario.

When the search button is clicked, user will be able to see various products listed (screen can be scrolled up and down). I need to write a program for clicking on any product (product could be on the 1st page or product could be anywhere -drag/dropdown).

I am looking for regular expressions with which I can identify the UI element uniquely and click on any one of them

example:
product name:dynex
model: dx0200
sku:34000

the above information will be changing dynamically, I need to write code for this.

could you please help me by providing the code or any other links to refer?

Thanks in advance
Siva

Question information

Language:
English Edit question
Status:
Answered
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
RaiMan (raimund-hocke) said : 		#1

So you first need to identify each item in the list.

Since this seems to be text, you have to first check, wether Region.text() feature reads the text correctly.
If this works, then you may have a chance, to solve this with Sikuli ;-)

Revision history for this message
Siva (avadhshiva) said : 		#2

Thanks for a quick turnaround RaiMan.

I have tried with the option given. In the Windows Mobile, I have created a region (On Sikuli - Create Region, captured that region).

reg=Region(here comes captured image of region)
myText=reg.text()
wait(2)
print myText

I was able to see the below results in the Sikuli

F7nd
 m?ion
I VS 1-10 EF $7 lgults
Best Match
2..1\|
lnsignia'? - 39" Class ? LCD -
10EOp - 60Hz - HDTV
**** ?k
on sane $279.99
AF
A AAI
19 reviews
Shipplng: Usually leavesourwarehouse in1 business d ay
32" Class / LCD/
Dynex??
~
*** 7k vk
On Sale $189.99
S4 reviews
Shlpplng: Usually le avesourwarehouse ln1 business day
19" Class/ LED /
Insignia'?
720p / 60Hz/ HDTV
i ink i ir
On Sale $99.99
67 reviews

For your information, on my Windows mobile application, the first image will have the below content
Insignia TM - 39" Class - LCD - 1080p - 60Hz - HDTV
Five Star symbols 19 reviews
On Sale #279.99
Shipping: Usually leaves our warehouse in 1 businessday

Region.Text() is capturing the information but how can I check whether one particular text is present on the region and how can I click on that UI element only (Please see the above Insignia information, say for example, I want to move up/down and click on any similar product on the windows mobile),

How can I do this?

Any help would be great!
Thanks
Siva

Revision history for this message
RaiMan (raimund-hocke) said : 		#3

I am willing to help you, but I am on vacation and without net the next days.
back home on monday.

Meanwhile have a look around in these Q&A's, there are many posts addressing all or parts of your problem.

Revision history for this message
RaiMan (raimund-hocke) said : 		#4

the only thing you can try to improve the OCR quality:

restrict the region to one line of text.

Then you might calculate other line-regions (using line height) and look for the text inside and compare it.

This is the only possibility, if you want to implement a robust approach (I do not recommend to use find("some text") becaus of the known problems/restrictions of text search)

If none of these works for you, then the current Sikuli is not the right tool for this purpose.

Can you help with this problem?

Provide an answer of your own, or ask Siva for more information if necessary.

To post a message you must log in.