On 28 April 2016 at 12:04, Daniel Thompson daniel.thompson@linaro.org wrote:
On 28/04/16 10:39, Naresh Kamboju wrote:
- Milosz
On 28 April 2016 at 13:36, Daniel Thompson daniel.thompson@linaro.org wrote:
On 28/04/16 05:32, Naresh Kamboju wrote:
On 28 April 2016 at 08:40, Leo Yan leo.yan@linaro.org wrote:
On Wed, Apr 27, 2016 at 09:27:37PM +0530, Naresh Kamboju wrote:
[ May be an off topic ]
Hi Leo,
As we discussed offline. I would like to be a part of EAS testing. Since I have Juno board on my desk it would be easy for me to run tests from tools like Workload automation and LISA.
Welcome :)
Milosz suggested to investigate the ways to automate this on LAVA like our team did for Workload automation [1].
If can use automatic method to launch EAS regression testing, this definitely will be very useful and save time.
The steps in Lisa's README.rd: $ source init_env $ nosetests -v tests/eas/rfc.py ./tools/report.py --base noeas --tests eas
Hi Leo,
Just curious, will you plan to enable this into wa2-lava.git?
Yes Because, adb and ssh connection frame work can be re-used from wa2-lava.git
Allowing us to run tests included with lisa (and other lisa derivations) would certainly be useful.
However, in addition, will it be possible for wa2-lava to generate lisa hacking session jobs as well? The automation used to construct KVM images from lisa repositories is useful for interactive work as well.
To explain a bit more...
One of the key features of lisa is that it can be used to interactively explore the effects of different synthetic workloads on a system (for example whilst seeking to reproduce "bad" behavior observed in a real system). Effectively it can act as a tool to help develop complex test cases, especially those that include energy metering.
At this point of time wa2-lava.git for full automation from end-to-end. The design was focused for automation only.
Download-> Configure -> and Install-tests Then running adb or ssh session to connect target device from KVM. Run The tests on target Collect the results on host parse the results on host Attach to LAVA job.
I only brought it up because there seemed to be more in common that different. Main differences are:
Instead of launching the test suite runner on KVM we need to connect to it interactively to conduct the hacking session. lisa uses the same devlib library that WA uses so the way it connects to the target device from KVM remains unaltered.
The results to be collected for the results bundle are different. The primary outputs are .ipynb files saved on the KVM filesystem. These record the interactive session and provide the code that can eventually flow into fully automatic tests.
'Interactive' doesn't go well with LAVA unless you're OK waiting to start your session for days. As example there are currently 136 jobs in juno queue. Some jobs take 24h to complete, but assuming it's only 1h for job with 8 active devices you would still wait for 17 hours before your session starts (and there is no guarantee).
What you describe above is a multinode hacking session. I have a code for such session for android. I guess it can be easily extended for linux targets as well. https://git.linaro.org/lava-team/hacking-session.git/blob/HEAD:/hacking-sess... https://git.linaro.org/lava-team/hacking-session.git/blob/HEAD:/hacking-sess...
For interactive way of running test is not in the scope now. I am adding Milosz for getting his thoughts on this subject.
To be clear, whilst it would be great to be able to reuse all the automatic VM construction and multi-job setup for interactive uses, fully automatic support is useful on its own.
However beware of release announcements declaring "we have added lisa support" without adding any caveats; that *would* imply support for interactive sessions.
I'm not sure I understand why the support for interactive session is implied (I didn't use the tool). For WA we only support certain use cases and provide means to extend the pool of use cases by users. If we start supporting lisa, it will be the same scenarion. Only some defined use case will be supported. Interactive session is out of scope here.
milosz
Daniel.