>150 test cases does not necessarily mean large - we have test jobs which
>produce 50,000 test cases per test job, that is when a large test set can
>become a problem, depending on the available resources on the server. It
>was these test jobs which led to the sections in the documentation which
>you mention later. If your server doesn't struggle with the current test
>jobs, you might not have anything to do at this stage.
>
>However, if you have clear groups of test cases, you should investigate
>using test sets which preserve those boundaries within the reported results:
>https://staging.validation.linaro.org/static/docs/v2/results-intro.html#test-set

Thanks for the information. Perhaps it would be a good idea to add these figures to the documentation, so that new users have an idea of what a “large” test set is.

I am just starting out with LAVA, so I haven’t set up any productive tests yet. At the moment I am creating a concept of how we can include LAVA into our workflow. Part of this is the question how we handle test jobs and how we store them in our SCM.

>MultiNode does offer a possible option but MultiNode itself is complex and
>not all test jobs would be able to use it - essentially you can use
>MultiNode with SSH to have multiple logins to the same device running the
>same software. Problems include issues with locking resources, load on the
>device from running tests in parallel, problems within the test being able
>to run in parallel in the first place amongst others.
>
>>From your enquiry, it does not sound as if you need any of that. You might
>want to look at a wrapper script which consumes the output of the tests and
>filters out noise but apart from that, 150 test cases is really not large.
>We routinely run functional tests which produce more test cases than that:
>https://staging.validation.linaro.org/results/214157
>https://staging.validation.linaro.org/scheduler/job/214157

I actually have to use MultiNode for some of our testcases anyway. These testcases need a remote server or client connected to the DUT (e.g. testing hardware interfaces like RS485, CAN, etc.).

And this is actually part of the question: When I declare all of my testcases in one test job, I have to declare the remote nodes for ALL of the tests in there as well. This makes the test job huge and confusing, though, I think. How do you handle such cases? Do you ever test that kind of interfaces at all?

Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks@garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!

Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz