KiCAD Source CI

Simon has something like this set up. He can probably speak to details. Are you looking to help out with some of the steps required for full testing?

I have a Jenkins instance at https://jenkins.simonrichter.eu/ that runs per-commit tests on Linux and Windows, as well as some nightly and weekly runs of the more expensive jobs. This also provides the signed Windows nightlies.

I would love to replace this with something with better performance and artifact handling (most of the time and SSD churn is spent on copying prerequisites and rebuilding things because prerequisites were copied and their modification date changed), but I haven't yet found a solution that really behaves better.

What would be especially welcome would be a solution that allows me to aggregate warnings and stack traces from different builds into a consistent view, taking into account that we can't build every commit in every configuration (e.g. Visual C++ /ANALYZE run takes five hours and parallelizes badly across CPUs), so it'd need to translate source locations of old warnings to the current state.

I am pretty new to kicad development, its just been about three days since i got the source code and today i managed to build it from source. But, I have been using it intensively for about 3 years. I have decided to contribute and ı thought it is logical to start from CI, but it can be improving tests, running full testing (somehow) or a better CI infrastructure.

Now i saw that you already have a jenkins for it.

Maybe as an initial step i can give CI plugins a try at least to compare the performance. But i am here for guidance if you already have items to be fixed on this topic.

Thanks Simon Richter, that solved my question.

I also tried to use Azure for fun, mostly because they support windows, linux and osx.

A dirty repo of my scripts are in https://github.com/nickoe/kicad-azure-test

It works quite well, but I was blocked becasue of a 10GB limitation via the free service.

Hi Again Folks,

So how about automated GUI tests? At least for openning problematic kicad_pcb files. As far as i saw the kicad tests are not that improved, except unit level tests. I am not sure if somebody working on tests but, i have created a draft by using WinApp driver(Selenium WebDriver like driver for windows apps) and Nunit(Just to manage tests), in my opinion that would be really beneficial to run them nightly.

You can check the initial draft here to get an overview:
https://github.com/cmbahadir/Kicad_GUI_Tests

It is just a fast protoype but if you think it is useful to have these tests, where imho it is, i am gonna improve it. And i hope it will make me to get familiar with kicad source and use r scenarios, gradually.

It will probably be easy to trigger them from a jenkins slave(Windows) nightly and report the results with each build.

Cons Are:

-- WinAppDriver is not a commonly supported tool so there are few bugs still.
-- It is only for Windows.
-- Test solution needs to be maintained separately.

Hi Cihan,

I'm not sure what the question is here but your testing solution does not cover all platforms because WinAppDriver and NUnit are windows only. This is not an acceptable solution for the project because we use the Boost unit testing framework as our test driver so if you want to contribute any of this to KiCad you will have to come up with a wxWidgets driver to do the UI testing with Boost. I seem to remember seeing a project to provide a wxWidgets UI testing framework but I cannot remember where I saw it.

Cheers,

Wayne

Have you looked into the wxWidgets UI testing framework (wxUIActionSimulator)? It is used for testing of the wxWidgets library that we use in their unit tests, and we have access to it through the wxWidgets dynamic library. It is designed for cross-platform testing.

The issue I see with the WinApp Driver route is that it is Windows only from what I can tell. Using the wxWidgets library would allow us to use the same unit tests for all platforms (or at least the ones that work, I have heard the OSX support might be iffy right now because Apple likes to break things with each OS update). These tests can then be integrated into our existing Boost testing framework.

I actually think that this discussion should be moved onto the developer mailing list, since more people might want to comment.

Ok, i haven't checked wxUIActionSimulator before, i will check and try to come up with a reliable gui test solution.

Actually, using WinApp driver was just the fastest way for me to test the users actions. And also from testers point of view, using wxUIActionSimulator may still be unit level test without running a separate exe and mimic users actions, but as i said i am not sure.

Thanks for the response.

Regards,
Mete