On 23 August 2016 at 18:40, Sandeep Chawla sandeep@cyngn.com wrote:

Hello,

Google has release the latest version of Tradefed with the Android N release.

https://source.android.com/devices/tech/test_infra/tradefed/index.html

Lots of dispatcher/slave features which LAVA already supports.

I'm not sure what you mean there. The support described on that page is not directly suitable for automation, requires specific tools to be available for specific jobs and also extra hardware. The multiple device support in the tool is actually at odds with what automation requires. The tool may be usable inside a test definition but the LAVA V2 support is independent of what this tool is trying to achieve.

Given this update, is LAVA exploring to adopt the new mechanism or continue developing its own architecture ?

0. There'll be no change in V1 support, that "android test architecture" is deprecated and is not receiving any development. 1: V2 support already isolates android testing inside LXC. 2: If the new tool(s) can be installed inside the LXC and use adb as currently then the existing V2 support can be used with updated test definitions. This puts the "new mechanism" inside the LXC and therefore there is no either | or decision. It is simply up to the test writer to incorporate the new tool into the LXC support. 3: The output of any tool used in a test definition needs to be mapped to lava-test-case to show up in the LAVA results - a custom wrapper script which understands the new tool would be part of the updated test definition. This would need to run inside the LXC and parse the output, just as happens with current adb output. 4: Features like power measurement are dependent on local admin setup - I can't say whether those could be supported at this stage. It depends entirely on how the tool itself works. There is not enough information to derive that yet. 5: Multiple devices would not be suitable for testing this way other than by using Multinode. The multiple device support in the tool would interfere with scheduling of LAVA devices used by other testjobs, so usage of the tool would need to be constrained to the one device assigned to that testjob. 6: Testing apps does not necessarily need LAVA unless those apps use hardware features which are only available on certain devices or only show bugs on specific types of devices. The Device OEM use case is the most likely to benefit from what LAVA can offer. 7: So far, there have been no requests to work on this from within the Linaro groups, so there is nobody working on this within Linaro so far as I am aware. It looks like the necessary work needs to only happen in the test definitions, so that is not a barrier to someone outside Linaro developing that support. 8: Overall, LAVA V2 does not have it's own architecture for these use cases other than the requirements which follow directly from automation. Once the complete process is automated then the tool would appear to fit inside the existing automation support using LXC, subject to the requirements involving external hardware.

Thanks Milosz and Neil for the feedback.

The reason I brought this up was , Tradefed provides some inbuilt tools(logging, log analyzer etc) for Android that need to be taken care by the test developer.

I think the best way to use Tradefed would be inside LXC in V2 as you suggested.

Some investigation is needed onre how best Tradefed can be used along with lava test case. There is the only overlap i see.

Once I migrate to V2, I will investigate this further at my end.

Thanks Sandeep

On Wed, Aug 24, 2016 at 3:17 AM, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:

Sandeep,

On 23 August 2016 at 18:40, Sandeep Chawla sandeep@cyngn.com wrote:

...

Hello,

Google has release the latest version of Tradefed with the Android N release.

https://source.android.com/devices/tech/test_infra/tradefed/index.html

Lots of dispatcher/slave features which LAVA already supports.

Thanks for pointing this out. I'll take a look.

...

Given this update, is LAVA exploring to adopt the new mechanism or

continue

...

developing its own architecture ?

I'm not sure there is overlap. Main purpose of LAVA is to provision the target device with your OS build. Tradefed doesn't have this implemented [1]. Testing comes later. We're already using tradefed for running CTS tests. If more can be done using similar means, that's great.

[1] Quote from the docs: "The OEM could implement a device flashing module that will execute during the Target Setup stage of the lifecycle. That module would have complete control over the device during its execution period, which would allow it to potentially force the device into the bootloader, flash, and then force the device to reboot back into userspace mode."

milosz