On 15 August 2017 at 23:26, Steve McIntyre steve.mcintyre@linaro.org wrote:
On Tue, Aug 15, 2017 at 01:07:16PM -0700, Arthur She wrote:
Hi Steve, Totally understand. My case is that we have several dummy HDMI dongles deployed to hikey 01, 02, 04 in staging LAVA. In order to verify/check if the HDMI dongles work, I have to make sure these hikey run the test job at least once.
This is an admin task, albeit you're doing the work on behalf of the admins. There are two ways to do this. Either use the admin tools to take devices offline temporarily so that there is a limited number of devices with the appropriate device tag of that device type or submit enough test jobs with the required device tag so that all devices run a test job.
HiKey devices always take a nominal amount of time to setup the LXC and do a simple boot test, so if there are 4 HiKey devices with suitable device tags, it's only a case of submitting 4 test jobs within the time that it takes one HiKey to complete one test job. LAVA scheduling will take care of running as many simultaneous test jobs as there are devices available.
If there is a possibility that the dongle might work most of the time but not all the time, then you will need to submit and monitor enough jobs that each device runs enough test jobs.
Make sure that all subsequent test jobs using the dongles do a minimal smoke test that the dongle works and include that in the results by explicitly calling lava-test-case dongle-check --result pass (or --result fail if your test has detected a failed dongle, just before the test shell exits).
This is not a normal use case as I mentioned and I totally agree with you regarding software testing.
Cool. :-)
Now I understand what you're trying to do - thanks for explaining some more!
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs