Hi Axel,
We have similar issues on our setup and there seem to be various root causes. In some cases, CTS/VTS test cases cause devices to get lost (rebooting, device becoming unresponsive etc.). In other cases, it's the infrastructure causing issues (unstable adb or USB connection etc.).
For your setup, is there a way to get the devices back without physically interacting with them? `adb kill-server` does sometimes magic. Do you know what exactly prevents the devices from being accessible via adb? Do they show up at all in `adb devices` or `lsusb`?
In general, CTS/VTS etc expect you to rerun your test sessions until you end up with a stable number of failures. From that perspective it is considered normal that after a single run you end up with incomplete modules or false positives. Regarding running CTS in LAVA, I implemented a variant of the Tradefed runner that works around some of the reasons for false positives and lost devices. It is more fault tolerant when it comes to lost (but hopefully recovering) devices. Also it automates the Tradefed retry mechanism. Have a look here, there are also example jobs: https://git.linaro.org/qa/test-definitions.git/tree/automated/android/multin... It does not implement VTS yet, but that should be reasonable simple to add.
Besides, the linked runner also implements sharding test runs across multiple devices by combining LAVA MultiNode jobs with adb TCP/IP connections. As it relies on a the DUTs having a network connection to the container running the Tradefed shell, it is not appropriate in all setups. Also, the network connection does not play well with some CTS modules (e.g., if network tests and modules test reboot devices, e.g. in CtsAppSecurityHostTestCases). However, you can use the runner on a single USB-attached DUT by setting the count of the "worker" role to 0 (in the example job yaml).
Karsten Tausche | Software Engineer Jollemanhof 17, 1019 GW Amsterdam, The Netherlands www.fairphone.com https://www.fairphone.com/en/
On Thu, Jun 13, 2019 at 4:37 PM Axel Lebourhis axel.lebourhis@linaro.org wrote:
Hi all,
I'm facing a pretty frustrating issue when running CTS/VTS with LAVA. I'm using Linaro's tradefed test definition : https://git.linaro.org/qa/test-definitions.git/tree/automated/android/tradef...
During some runs, the adb connection is lost, leading to incomplete test job. Do you know if this behavior is known and mostly general ? Or is it a bad configuration on my side ? Maybe someone knows some way to keep a reliable adb connection to the target ?
Best regards, Axel _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users