SikuliX

Is there any interest in an Open Source Java Automation Framework that uses Sikuli?

Asked by someotherperson432423423

Heya Sikuli people! Just gotta say, Sikuli has saved my bacon quite a bit in the past. I helped leveraged it as the image search part of an automation framework that was able to automate flash games and it just made things work beautifully. I left that job and just acquired another where they want me to basically do the same thing.

I've always felt pretty indebted to Sikuli, it's developers, and it's users (Especially Raiman who helped me out on the job more times then he knows). So I asked my bosses if it'd be possible to open source the automation framework that I'm writing and they were actually pretty warm to the idea. It may have to be for non-commercial uses to cover our butts but that's still pretty awesome right?

I've only done a few lazy research days designing and hacking at it since it's not the only thing I have to do at work, but to give you a taste of what I've coded up before and what this framework will need to do here's a little overview.

The general goal of the framework is to use allow us to generically automate the test cases for any of our games. Using Sikuli's image searching I've been able to arbitrarily automate just about any game or product so that's obviously not too much of a problem. You can really do all that in the IDE, this framework is really just going to be a lot of nice features to build/maintain/test large amounts of scripts like this that someone who needs to automate a big product would need.

It needs to be pretty easy to run tests across lots of platforms so it's going to have a lot of logic about that, once again going to use different directories of images to intelligently navigate around with logic to pop open Virtual Machines with configurations we want to run tests on.

It's going to eventually need some good logging, currently I have a "Take a screenshot and shove in log folders!" function which is okay for seeing what's going on for now, but we're thinking of shoving some video recording functionality into the framework and just making videos of every script as it runs on the computer.

Besides the video logging and grabbing images from the web I've already made up a framework that does all this, and I'm doing it again now. But what else could it do? That's part of the reason I'd like to get it into the hands of ya'll.

Anyhow, to go a bit deeper here's some packages I've got currently in the prototype to explain some high level structure ideas.

images: I've currently got a nice organized directory file for storing all the images, but an interesting feature might be later to hook all images up to things on live webservers. Doing that we could link images in scripts to web files, and when that file gets updated, so does the script! This could reduce image maintenance a whole lot.

Library: The crux of the framework will be how useful all these nice little functions will be. I found that I could automate my games with almost all my calls (90%+) with JUST wait() and click(). Also going to be putting the bells and whistles here, like navigating to the right product to test, making sure you're on the right test configuration, recovery from errors or popups/abnormal behavior, logging and error handling. I'm also having product-specific library files that extend the main library file so you can have product-specific functions that don't bloat up the main file.

You could then run your tests like this: OS:windows/browser:firefox/suite:someTestSuite/game:Awesomegame100/scriptname

products: a package that contains packages for all your product test suites and test cases

products.someproduct.suites: I'm using JUnit to organize my scripts and tests in the hopes that it'll simplify organizing and test case reporting. So far it's looking pretty good. Anyhow, at this level you could have a collection of all your test suites for your product that you could run at any time.

products.someproduct.testscripts: this package contains a bunch of java files that all have JUnit test cases in them. Each test case is really just a bunch of sikuli calls that automate a piece of functionality, in my case for a game.

So I certainly haven't gotten too far, but I've built it before and I'm going to build it either way. I'd just much rather have it in the hands of others because I really believe in Open Source. I know for a fact lots of big companies have all reinvented the wheel and done all of this for themselves, I helped one of them. I think that's a damned shame and this might be a good opportunity to give a powerful tool to everyone. As for next steps I don't know. I asked a knowledgeable person who said a github/opensource organization project like launchpad or sourceforge would work. I could also maybe have a little branch here on the Sikuli launchpad if the Sikuli developers are okay with this little mini-project. I'm not too entirely sure what options to take, I could sure use some opinions on that.

Hey TLDR? Thanks for reading. If you got any feedback/questions/comments I'll be checking this often. I hope this idea/tool can really help some other people. I'd hate to keep it all to myeslf, again.

Question information

Language:
English Edit question
Status:
Solved
For:
SikuliX Edit question
Assignee:
No assignee Edit question
Solved by:
someotherperson432423423
Solved:
Last query:
Last reply:
Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#1

Wow I'm a ditz. I guess the QUESTION is: Would people be interested in having an open source automation framework in Java that integrates Sikuli with the features and description above?

And if so, can I get some advice/help on how to go about doing that?

Revision history for this message
Tsung-Hsiang Chang (vgod) said : 		#2

Sounds interesting. Can you give more details on how the API of your framework looks like?

Actually we are working on an extension framework in Sikuli Script so developers and hackers can easily build libraries on top of Sikuli. The extension framework will released with X1.0-RC2. You are very welcome to develop this testing framework as a Sikuli extension.

I notice some features you mentioned are already in Sikuli or under development. For example, the remote image library is supported in X 1.0-RC1. You can add a URL with addImagePath, so Sikuli will try to fetch all images from that URL. With this technique, you can switch between different (local/remote) image libraries without modifying each images in the script.
Screenshot logging is a planned feature. So you don't really need to implement it.

We really want to see more libraries/frameworks are developed based on Sikuli. If you feel you are doing something that should be built into the Sikuli core, just let us know and read this document http://sikuli.org/docx/contributing.html to learn how to submit patches.

Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#3

Ah yes, the feature I mentioned wasn't going to be implemented I mentioned it because it's awesome and the framework could use it (In reference to the web images thing).

What I'm building probably won't be a part of sikuli core, it'll just use that sexy little jar that lets me use Sikuli in the java code. It would itself be a lightweight framework for organizing, making, running, and maintaining the type of automation tests I described. If you want I'd be happy to send you a very very minimal prototype I've built in my free time the last few days. I'm sure it'd answer more questions pretty quick.

Revision history for this message
Tsung-Hsiang Chang (vgod) said : 		#4

It would be good if you have a prototype. Feel free to post here and I'm sure many people would be interested.

Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#5

Cool. Here's a ridiculously minimal prototype with a few library functions, exploratory code, and general package structure. It requires you to follow the guide on the Sikuli site to making Sikuli available in your Java code. If you wanna try and use it I'd suggest making a new project in eclipse with "Java" as the root directory and linking the Sikuli.jar as an external Jar. Should boot it right up. Might not be enough to be impressive yet, if so I'll just hammer on it til it's getting as good as the last one I helped build.

Enjoy:

https://docs.google.com/leaf?id=0BwZ6aoCy8tbrNWQwYzhhZTMtNjc0ZC00NTAzLWEzYTMtYTkyMDUyNTBhOTAz&hl=en&authkey=CLGA-Rw

Revision history for this message
Tsung-Hsiang Chang (vgod) said : 		#6

@Topagae,

I just take a look into your code and have a better idea of what you are trying to do now. I think to build a framework based on JUnit is a good idea. We also want to have a better support for JUnit in Sikuli, however, we will do this for Sikuli scripts (Jython) and won't conflict with your framework.

It would be really cool if you can build a generic framework for testing a product on different platforms with VMs. Given your project would be developer oriented, I think you can host your code on github.

Good luck. I'm looking forward to it.

Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#7

Well I sure hope I can build it again, I've done it before! This one'll just be open source ;3. I'll look into Github. I'll probably be posting an early Alpha sometime next week as I got to get ready to show it to some superiors. Presuming it works out nice I'll just shove it up as that and continue to develop it on my own.

I'll post a link here later when I do. Have a good one peoples.

Revision history for this message
surfdork (surfdork) said : 		#8

Topagae,

Very cool, I'd like to help (if I can).

I've built a framework around Sikuli used for testing a videoconferincing application with an emphasis on event driven testing.

Thanks to how cool Xrc2 is , I'm going to rewrite the entire framework utilizing the new features.
The best new feature (for my purposes) is the new file path that uses URL's
This means that I can keep an image library in mySQL as a blob with all requisite metadata in the same record.
No more use of a file structure to store images and another location to store the TOF's I was able to pitch this at work thus ressurecting my project for another month.

I wil eliminate use of the configparser as well which is a never ending headache when you are managing 200+ separate images.

Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#9

Atm, I'm forced to just keep using the directory structure, but that's another improvement for another time. I'm going try and see if my bosses will let me put it on github X_X

Revision history for this message
daluu (cuuld) said : 		#10

On a related note, with exception of new ideas & features, I tend to think reinventing the wheel is rather a partial waste of resources.

Wherever possible, it would also be wise to see how Sikuli might be integrated into "existing" automation frameworks as well as how one's organization/employer might be able to leverage existing frameworks as well, that latter idea though only works best for those that have no existing automation framework, because it will be hard to try and move off one framework for another. But no big deal if you either have to build from scratch or pick an existing framework to extend on top of, unless the organization has concerns over OSS licensing (which generally shouldn't matter unless they are in the business of selling automation frameworks and tools).

For some current open source suggestions for integrating Sikuli with:

* RobotFramework.org
* jsystemtest.org
* staf.sourceforge.net

I've already built a basic wrapper library for RobotFramework as well as command line API tool for Sikuli all in one. With command line API, no need to write code or scripts just to use Sikuli. Since Sikuli's command line interface doesn't offer option to execute (API) commands, that's what I sought to provide a solution for.

http://code.google.com/p/simplesikuli/

It's on Google code right now, but if there's demand, I can move it to GitHub instead.

Revision history for this message
someotherperson432423423 (someotherperson432423423-deactivatedaccount) said : 		#11

Wow this is an old one. Already done though. My previous employer (Zynga) uses Sikuli as a wrapper for an automation framework they license. I wrote a new one for the employer I'm working for now. My company sadly doesn't want to make it open source. Typical