Hello Lava-users,
I am trying to run some test suites in LAVA and my test suite is organized as below -single script with multiple test cases -want to have report the results from LAVA of all test cases -don't want to depend on the return value of script which will make as test case as PASS or FAIL
Developing test suites in shell script and using sh-test-lib and attached is sample test script for reference. Want to report the result of iptable_packageCheck_test and iptable_kernel_config_test separately in LAVA framework and will be having single script for both the test cases.
Based on above criteria which is best method of running the test script in LAVA.Referred https://validation.linaro.org/static/docs/v2/developing-tests.html
Following is the content of LAVA Test Shell Definition I am using
metadata: name: applications format: "Lava-Test-Shell Test Definition 1.0" description: "Application tests." os: - debian scope: - Functional environment: - lava-test-shell run: steps: - cd /opt/applications - ./cron.sh - ./iptable.sh - . ../utils/send-to-lava.sh output/result.txt
Thanks, Hemanth.
On Mon, 29 Oct 2018 at 05:54, Hemanth K V kv.hemanth.mys@gmail.com wrote:
Hello Lava-users,
I am trying to run some test suites in LAVA and my test suite is organized as below -single script with multiple test cases
-want to have report the results from LAVA of all test cases -don't want to depend on the return value of script which will make as test case as PASS or FAIL
Developing test suites in shell script and using sh-test-lib and attached is sample test script for reference. Want to report the result of iptable_packageCheck_test and iptable_kernel_config_test separately in LAVA framework and will be having single script for both the test cases.
Based on above criteria which is best method of running the test script in LAVA.Referred https://validation.linaro.org/static/docs/v2/developing-tests.html
Script needs to check the lav-test-case in $PATH Script executes the calls and parses the output / return code to calculate the result of that call. Script calls lava-test-case without using the --shell option LAVA Test Shell Definition calls the script instead of lava-test-case.
https://master.lavasoftware.org/static/docs/v2/writing-tests.html
https://master.lavasoftware.org/static/docs/v2/writing-tests.html#write-port...
Following is the content of LAVA Test Shell Definition I am using
metadata: name: applications format: "Lava-Test-Shell Test Definition 1.0" description: "Application tests." os: - debian scope: - Functional environment: - lava-test-shell run: steps: - cd /opt/applications - ./cron.sh - ./iptable.sh - . ../utils/send-to-lava.sh output/result.txt
Probably not. Expand iptable.sh to call lava-test-case --result as soon as the test operation has completed. Do not wait until everything is done to parse some log file because that leaves you open to getting no results at all if any step prior to the parsing should fail.
Thanks, Hemanth.
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
On Mon, Oct 29, 2018 at 2:22 PM Neil Williams neil.williams@linaro.org wrote:
On Mon, 29 Oct 2018 at 05:54, Hemanth K V kv.hemanth.mys@gmail.com wrote:
Hello Lava-users,
I am trying to run some test suites in LAVA and my test suite is organized as below -single script with multiple test cases
-want to have report the results from LAVA of all test cases -don't want to depend on the return value of script which will make as test case as PASS or FAIL
Developing test suites in shell script and using sh-test-lib and attached is sample test script for reference. Want to report the result of iptable_packageCheck_test and iptable_kernel_config_test separately in LAVA framework and will be having single script for both the test cases.
Based on above criteria which is best method of running the test script in LAVA.Referred https://validation.linaro.org/static/docs/v2/developing-tests.html
Script needs to check the lav-test-case in $PATH Script executes the calls and parses the output / return code to calculate the result of that call. Script calls lava-test-case without using the --shell option LAVA Test Shell Definition calls the script instead of lava-test-case.
https://master.lavasoftware.org/static/docs/v2/writing-tests.html
https://master.lavasoftware.org/static/docs/v2/writing-tests.html#write-port...
Following is the content of LAVA Test Shell Definition I am using
metadata: name: applications format: "Lava-Test-Shell Test Definition 1.0" description: "Application tests." os: - debian scope: - Functional environment: - lava-test-shell run: steps: - cd /opt/applications - ./cron.sh - ./iptable.sh - . ../utils/send-to-lava.sh output/result.txt
Probably not. Expand iptable.sh to call lava-test-case --result as soon as the test operation has completed. Do not wait until everything is done to parse some log file because that leaves you open to getting no results at all if any step prior to the parsing should fail.
I see the above test is using the same approach as https://git.linaro.org/qa/test-definitions.git/tree/automated. With this approach, we avoid to use lava-specific helpers in test script so that the script works without LAVA. When running in LAVA, 'send-to-lava.sh' uses lava-test-case to send results generated by test script to LAVA. As Neil pointed, the problem is it doesn't benefit from LAVA V2's live reporting features.
I think it is possilble to run 'send-to-lava.sh' in background and send new lines added to result.txt instantly, but that is risky as it may prevent test action from exiting when the background process not stopped/killed properly. From my point of view, tests require live reporting should use 'parse parttern' instead, for example https://git.linaro.org/qa/test-definitions.git/tree/automated/linux/kselftes...
Chase
Thanks, Hemanth.
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
lava-users@lists.lavasoftware.org
-
Chase Qi -
Hemanth K V -
Neil Williams