Scripts recording tool do you need one?

Hi,

How does the picture capturing work? How can you automatically decide what is the area of the image you are clicking?
Or do you have to define that area manually? If so, how is it different from the built in IDE functionality?

Thanks

Hi Eugene!
You should define area of the image, similar to IDE.
How it differs, let see,
1. It records all commands in java format. So you don't need to define extra methods in you class (such as click, doubleclick, etc) or extends Screen class. You don't need to think about semicolons;
2. It records both the pictures and your input, so, you don't need to alt-tab to the IDE to insert a value.
3. You can run it when you're doing positive testing.
4. Every Run you can define particular folders for images and script file, or leave the previous choice.

Honestly, I don't know all capabilities of the IDE, but I know that it cannot capture a picture all the time when you're clicking :)

@Denis
to completely revise the initial capturing process and to have some companion at runtime to step through a workflow and modify/optimize the captured images (some floating popup) is on the list for version 2
This will be a separate feature, to be able to integrate it as plugin into other IDE's or just use it standalone as a "workflow recorder"

If you have any contribution: always welcome.

The place to get in direct contact with me and my stuff: http://sikulix.com

made it a request bug.

please add related comments now to the bug thread.

Hi all,
I have been using for several years a recorder which captures the images and puts the actions into the workflow, but finally I found that typing scripts by hands in Sikuli is more efficient, because:
1. Buttons and other screen elements almost always get highlighted and become different when the mouse is over them. Moreover, in most remote desktops, the mouse is drawn on the screenshots. Therefore the recorded images are too often useless for playback with search of the images on the screen.
2. Even if not highlighted, the images are captured without any sense, named mostly by time, and placed into the scripts by their names. When you have several dozens of test scripts, the system becomes non-maintainable: when programmers change an icon on one button, you have to update manually every script. It's even often easier to record them again than to update. The problem can be solved if you have GUI map of common images and provide that the scripts refer to the same images when you click them during the record, but it will require image recognition capability from the recorder, that can really become a large task. Instead, if you write the script manually, you can do it easily, so that you can change only one image when something changes in the tested application.
3. Manually you write clever scripts, finding screen contents by relative positions with left(), above() etc., so that the scripts survive future GUI changes occurring during software development. But with the automatic recorder, it's quite difficult to provide that it understands what spatial relations you mean. It's still possible to record the actions and them to edit the script manually, inserting necessary logics, but in this case the recording itself does not save significant time.

So, I suggest, writing a practically useful recorder can require much more work than it seems initially.

Hi Eugene!

I agree with all points that you mentioned, but let me share why I found the script writer extremely useful. By now, I'm using it simply to take pictures. It has significantly reduced my time to capture the required images. The panel is always on top of my screen and when I need to take some screenshots I just press Record button and start cutting. When I press Stop I get all my clicks written in script and all my images stored in the particular place. Then I copy the commands to my main script and continue coding.
I save time by not using a snipping tool (especially for linux there is no such handy tool like for windows), not think about the images names, not choosing the directory to save the images and not writing the click commands.

@Denis & Eugene
thanks for your valuable input.
additional aspect: there are surely different requirements for the use in a visual testing environment or for some automation.

For automation purposes you need reliable scripts, that even take care for some special situations, without crashing.
In testing it is ok, to just let a test fail, if something is not as it should be.

Automation scripts usually are tailored to solve a specific problem, usually for a fixed environment.
In testing today, the tests have to cover many system and rendering (mainly with browsers) environments.

In automation you want to be sure, that after some workflow step is processed, the intended operational result is achieved.
in testing usually the sequence is: when I do this, the GUI should look like that afterwards. And GUI testing often is separated from operational, data related testing (for which testers will surely not use a tool like SikuliX).

So a SikuliX recorder should surely support a sophisticated image capturing (naming, organising, optimisation, variants, ...).
With a little effort, this is possible already today:
- use the capture hotkey, which in most cases allows to capture the GUI elements in a way, as they are shown at runtime
- switch on the manual naming, so you can implement a naming convention from beginning
- separate code from images (code in one .sikuli, images, patterns and other image related information in other .sikuli), so you can decide at runtime which one to use via import

In my opinion it would be a step in the right direction, to force the user, to first decide for the element's name and some attributes (only image and/or an element that might be acted on by mouse and/or keyboard, possible clickpoint relation to other images already in the stock, ....) and then capture the image and store it in an organised way (image groups).

I will copy these comments to the related bug and would really appreciate to add further comments there:
bug 1434434