Hello,
we have the need to perform tests that require reboots of the DUT between their executions. Few examples are to check rootfs upgrades or to check configurations changes to persist .
I have few questions: * Does LAVA support those cases? * If yes, does LAVA support multiple reboots? * If yes, how can I write tests in order to run different sets of tests at any boot. * Example: 1) do an upgrade 2) reboot the device 3) Check if the upgrade was successful * How can I structure my pipeline?
Thanks
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Tue, 22 Jan 2019 at 14:28, Diego Russo Diego.Russo@arm.com wrote:
Hello,
we have the need to perform tests that require reboots of the DUT between their executions. Few examples are to check rootfs upgrades or to check configurations changes to persist .
I have few questions:
- Does LAVA support those cases?
This is board dependent and, in some cases, deployment method dependent as well. TFTP deployments, for example, will need a redeployment.
- If yes, does LAVA support multiple reboots?
Specify the actions in the test job actions list:
deploy, boot, test, boot, test
- If yes, how can I write tests in order to run different sets of tests at any boot.
- Example: 1) do an upgrade 2) reboot the device 3) Check if the upgrade was successful
Specify a boot action to control how the device gets rebooted.
- How can I structure my pipeline?
The hikey 6220 health check deploys two different systems, rebooting appropriately: https://staging.validation.linaro.org/scheduler/job/247930/definition
Also https://staging.validation.linaro.org/scheduler/job/246536/definition
Thanks
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
On 22 Jan 2019, at 14:32, Neil Williams <neil.williams@linaro.orgmailto:neil.williams@linaro.org> wrote:
On Tue, 22 Jan 2019 at 14:28, Diego Russo <Diego.Russo@arm.commailto:Diego.Russo@arm.com> wrote:
Hello,
we have the need to perform tests that require reboots of the DUT between their executions. Few examples are to check rootfs upgrades or to check configurations changes to persist .
I have few questions: * Does LAVA support those cases?
This is board dependent and, in some cases, deployment method dependent as well. TFTP deployments, for example, will need a redeployment.
For the deploy action I have uboot-ums and a tftp.
* If yes, does LAVA support multiple reboots?
Specify the actions in the test job actions list:
deploy, boot, test, boot, test
I’ve just tried but after the execution of the test, when it goes in to boot again, it redeploys the board again. I just want a simple reboot of the board.
* If yes, how can I write tests in order to run different sets of tests at any boot. * Example: 1) do an upgrade 2) reboot the device 3) Check if the upgrade was successful
Specify a boot action to control how the device gets rebooted.
That’s a good point. I need to reboot WaRP7 and a RPi3 boards. And if I replicate the boot section it will deploy again the boardd. How can I skip the redeployment
* How can I structure my pipeline?
The hikey 6220 health check deploys two different systems, rebooting appropriately: https://staging.validation.linaro.org/scheduler/job/247930/definition
Also https://staging.validation.linaro.org/scheduler/job/246536/definition
Thanks
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.orgmailto:Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
--
Neil Williams ============= neil.williams@linaro.orgmailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On 22 Jan 2019, at 15:35, Diego Russo Diego.Russo@arm.com wrote:
On 22 Jan 2019, at 14:32, Neil Williams neil.williams@linaro.org wrote:
On Tue, 22 Jan 2019 at 14:28, Diego Russo Diego.Russo@arm.com wrote:
Hello,
we have the need to perform tests that require reboots of the DUT between their executions. Few examples are to check rootfs upgrades or to check configurations changes to persist .
I have few questions:
- Does LAVA support those cases?
This is board dependent and, in some cases, deployment method dependent as well. TFTP deployments, for example, will need a redeployment.
For the deploy action I have uboot-ums and a tftp.
I made some progress with this. Apparently there is a “minimal" boot actiom which is not documented.
https://git.lavasoftware.org/lava/lava/blob/master/lava_dispatcher/actions/b...
It connects and reset the DUT. This is exactly what I needed.
We encountered another problem though.
My pipeline is something like this
- boot: method: u-boot commands: ums auto_login: login_prompt: '(.*) login:' username: 'root' prompts: - 'root@(.*):~#' timeout: minutes: 50
- test: timeout: minutes: 50 definitions: … … …
- boot: method: minimal auto_login: login_prompt: '(.*) login:' username: 'root' prompts: - 'root@(.*):~#' timeout: minutes: 50
When it boots the second time for some reasons it matches straightaway the first prompt. Here a copy/paste of the log
- {"dt": "2019-01-22T16:13:08.561407", "lvl": "results", "msg": {"case": "pdu-reboot", "definition": "lava", "duration": "1.05", "extra": !!python/object/apply:collections.OrderedDict [[["status", "success"]]], "level": "4.2.1", "namespace": "common", "result": "pass"}} - {"dt": "2019-01-22T16:13:08.561840", "lvl": "debug", "msg": "end: 4.2 reset-device (duration 00:00:01) [common]"} - {"dt": "2019-01-22T16:13:08.562100", "lvl": "debug", "msg": "start: 4.3 auto-login-action (timeout 00:49:59) [common]"} - {"dt": "2019-01-22T16:13:08.562389", "lvl": "debug", "msg": "Using line separator: #'\n'#"} - {"dt": "2019-01-22T16:13:08.562589", "lvl": "info", "msg": "Waiting for the login prompt"} - {"dt": "2019-01-22T16:13:08.562773", "lvl": "info", "msg": "Parsing kernel messages"} - {"dt": "2019-01-22T16:13:08.562940", "lvl": "debug", "msg": ["-+\[ cut here \]-+\s+(.*\s+-+\[ end trace (\w*) \]-+)", "(Unhandled fault.*)\r\n", "Kernel panic - (.*) end Kernel panic", "Stack:\s+(.*\s+-+\[ end trace (\w*) \]-+)", "root@(.*):~#", "(.*) login:", "Login incorrect"]} - {"dt": "2019-01-22T16:13:08.563215", "lvl": "debug", "msg": "[auto-login-action] Waiting for messages, (timeout 00:49:59)"} - {"dt": "2019-01-22T16:13:08.563600", "lvl": "debug", "msg": "Matched prompt #4: root@(.*):~#"} - {"dt": "2019-01-22T16:13:08.563932", "lvl": "results", "msg": {"case": "kernel-messages", "definition": "lava", "duration": "0.00", "extra": {"extra": [{"success": "root@(.*):~#"}]}, "level": "4.3", "namespace": "common", "result": "pass"}} - {"dt": "2019-01-22T16:13:08.564717", "lvl": "debug", "msg": "Sending username root"} - {"dt": "2019-01-22T16:13:08.565048", "lvl": "input", "msg": "root\n"} - {"dt": "2019-01-22T16:13:08.667599", "lvl": "target", "msg": "root@mbed-linux-os-1883:~# root"} - {"dt": "2019-01-22T16:13:08.668237", "lvl": "debug", "msg": "auto-login-action: Wait for prompt ['root@(.*):~#', 'Login incorrect', 'Login timed out'] (timeout 00:49:59)"} - {"dt": "2019-01-22T16:13:08.996556", "lvl": "target", "msg": "\0NOTICE: BL2: v1.5(release):v1.5-829-g36044ba"} - {"dt": "2019-01-22T16:13:09.054952", "lvl": "target", "msg": "NOTICE: BL2: Built : 04:58:25, Jan 22 2019”}
The workaround I’m implementing is to change the prompt before the reboot and wait for the modified prompt afterwards but for some reason this doesn’t work. Basically I create a .profile which modifies PS1.
LAVA gets to a point where it sends the username but then it doesn’t match the prompt.
- {"dt": "2019-01-23T19:10:42.315759", "lvl": "target", "msg": "Mbed Linux OS mbl-os-0.5.120 mbed-linux-os-8881 /dev/ttymxc0"} - {"dt": "2019-01-23T19:10:42.316047", "lvl": "target", "msg": ""} - {"dt": "2019-01-23T19:10:42.317161", "lvl": "debug", "msg": "Matched prompt #5: (.*) login:"} - {"dt": "2019-01-23T19:10:42.317434", "lvl": "results", "msg": {"case": "kernel-messages", "definition": "lava", "duration": "65.89", "extra": {"extra": [{"success": "(.*) login:"}]}, "level": "4.3", "namespace": "common", "result": "pass"}} - {"dt": "2019-01-23T19:10:42.317829", "lvl": "debug", "msg": "Sending username root"} - {"dt": "2019-01-23T19:10:42.318073", "lvl": "input", "msg": "root\n"} - {"dt": "2019-01-23T19:10:42.418820", "lvl": "target", "msg": "mbed-linux-os-8881 login: root"} - {"dt": "2019-01-23T19:10:42.419545", "lvl": "debug", "msg": "auto-login-action: Wait for prompt ['root@(.*)-second-stage:~#', 'Login incorrect', 'Login timed out'] (timeout 00:48:53)"} - {"dt": "2019-01-23T19:10:42.423096", "lvl": "target", "msg": "root”}
This is what I have in ser2net if I boot the board manually
Mbed Linux OS mbl-os-0.5.120 mbed-linux-os-8881 /dev/ttymxc0
mbed-linux-os-8881 login: root root@mbed-linux-os-8881-second-stage:~$
Which looks OK to me.
Any idea why the second prompt matching is not working?
Cheers
- If yes, does LAVA support multiple reboots?
Specify the actions in the test job actions list:
deploy, boot, test, boot, test
I’ve just tried but after the execution of the test, when it goes in to boot again, it redeploys the board again. I just want a simple reboot of the board.
- If yes, how can I write tests in order to run different sets of tests at any boot.
- Example: 1) do an upgrade 2) reboot the device 3) Check if the upgrade was successful
Specify a boot action to control how the device gets rebooted.
That’s a good point. I need to reboot WaRP7 and a RPi3 boards. And if I replicate the boot section it will deploy again the boardd. How can I skip the redeployment
- How can I structure my pipeline?
The hikey 6220 health check deploys two different systems, rebooting appropriately: https://staging.validation.linaro.org/scheduler/job/247930/definition
Also https://staging.validation.linaro.org/scheduler/job/246536/definition
Thanks
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Wed, 23 Jan 2019 at 19:35, Diego Russo Diego.Russo@arm.com wrote:
On 22 Jan 2019, at 15:35, Diego Russo Diego.Russo@arm.com wrote:
On 22 Jan 2019, at 14:32, Neil Williams neil.williams@linaro.org wrote:
On Tue, 22 Jan 2019 at 14:28, Diego Russo Diego.Russo@arm.com wrote:
Hello,
we have the need to perform tests that require reboots of the DUT between their executions. Few examples are to check rootfs upgrades or to check configurations changes to persist .
I have few questions:
- Does LAVA support those cases?
This is board dependent and, in some cases, deployment method dependent as well. TFTP deployments, for example, will need a redeployment.
For the deploy action I have uboot-ums and a tftp.
I made some progress with this. Apparently there is a “minimal" boot actiom which is not documented.
https://git.lavasoftware.org/lava/lava/blob/master/lava_dispatcher/actions/b...
It connects and reset the DUT. This is exactly what I needed.
We encountered another problem though.
My pipeline is something like this
- boot: method: u-boot commands: ums auto_login: login_prompt: '(.*) login:' username: 'root' prompts: - 'root@(.*):~#'
This prompt is too generic, it looks like you've copied it from the LXC support where it is needed because the prompt changes according to test job specific data. Use a prompt without wildcards.
timeout: minutes: 50
- test: timeout: minutes: 50 definitions: …
… …
- boot: method: minimal auto_login: login_prompt: '(.*) login:' username: 'root' prompts:
timeout: minutes: 50
- 'root@(.*):~#'
When it boots the second time for some reasons it matches straightaway the first prompt. Here a copy/paste of the log
pexpect has matched something in the output based on the wildcard.
- {"dt": "2019-01-22T16:13:08.561407", "lvl": "results", "msg": {"case": "pdu-reboot", "definition": "lava", "duration": "1.05", "extra": !!python/object/apply:collections.OrderedDict [[["status", "success"]]], "level": "4.2.1", "namespace": "common", "result": "pass"}}
- {"dt": "2019-01-22T16:13:08.561840", "lvl": "debug", "msg": "end: 4.2 reset-device (duration 00:00:01) [common]"}
- {"dt": "2019-01-22T16:13:08.562100", "lvl": "debug", "msg": "start: 4.3 auto-login-action (timeout 00:49:59) [common]"}
- {"dt": "2019-01-22T16:13:08.562389", "lvl": "debug", "msg": "Using line separator: #'\n'#"}
- {"dt": "2019-01-22T16:13:08.562589", "lvl": "info", "msg": "Waiting for the login prompt"}
- {"dt": "2019-01-22T16:13:08.562773", "lvl": "info", "msg": "Parsing kernel messages"}
- {"dt": "2019-01-22T16:13:08.562940", "lvl": "debug", "msg": ["-+\[ cut here \]-+\s+(.*\s+-+\[ end trace (\w*) \]-+)", "(Unhandled fault.*)\r\n", "Kernel panic - (.*) end Kernel panic", "Stack:\s+(.*\s+-+\[ end trace (\w*) \]-+)", "root@(.*):~#", "(.*) login:", "Login incorrect"]}
- {"dt": "2019-01-22T16:13:08.563215", "lvl": "debug", "msg": "[auto-login-action] Waiting for messages, (timeout 00:49:59)"}
- {"dt": "2019-01-22T16:13:08.563600", "lvl": "debug", "msg": "Matched prompt #4: root@(.*):~#"}
- {"dt": "2019-01-22T16:13:08.563932", "lvl": "results", "msg": {"case": "kernel-messages", "definition": "lava", "duration": "0.00", "extra": {"extra": [{"success": "root@(.*):~#"}]}, "level": "4.3", "namespace": "common", "result": "pass"}}
- {"dt": "2019-01-22T16:13:08.564717", "lvl": "debug", "msg": "Sending username root"}
- {"dt": "2019-01-22T16:13:08.565048", "lvl": "input", "msg": "root\n"}
- {"dt": "2019-01-22T16:13:08.667599", "lvl": "target", "msg": "root@mbed-linux-os-1883:~# root"}
- {"dt": "2019-01-22T16:13:08.668237", "lvl": "debug", "msg": "auto-login-action: Wait for prompt ['root@(.*):~#', 'Login incorrect', 'Login timed out'] (timeout 00:49:59)"}
- {"dt": "2019-01-22T16:13:08.996556", "lvl": "target", "msg": "\0NOTICE: BL2: v1.5(release):v1.5-829-g36044ba"}
- {"dt": "2019-01-22T16:13:09.054952", "lvl": "target", "msg": "NOTICE: BL2: Built : 04:58:25, Jan 22 2019”}
The workaround I’m implementing is to change the prompt before the reboot and wait for the modified prompt afterwards but for some reason this doesn’t work. Basically I create a .profile which modifies PS1.
LAVA gets to a point where it sends the username but then it doesn’t match the prompt.
- {"dt": "2019-01-23T19:10:42.315759", "lvl": "target", "msg": "Mbed Linux OS mbl-os-0.5.120 mbed-linux-os-8881 /dev/ttymxc0"}
- {"dt": "2019-01-23T19:10:42.316047", "lvl": "target", "msg": ""}
- {"dt": "2019-01-23T19:10:42.317161", "lvl": "debug", "msg": "Matched prompt #5: (.*) login:"}
- {"dt": "2019-01-23T19:10:42.317434", "lvl": "results", "msg": {"case": "kernel-messages", "definition": "lava", "duration": "65.89", "extra": {"extra": [{"success": "(.*) login:"}]}, "level": "4.3", "namespace": "common", "result": "pass"}}
- {"dt": "2019-01-23T19:10:42.317829", "lvl": "debug", "msg": "Sending username root"}
- {"dt": "2019-01-23T19:10:42.318073", "lvl": "input", "msg": "root\n"}
- {"dt": "2019-01-23T19:10:42.418820", "lvl": "target", "msg": "mbed-linux-os-8881 login: root"}
- {"dt": "2019-01-23T19:10:42.419545", "lvl": "debug", "msg": "auto-login-action: Wait for prompt ['root@(.*)-second-stage:~#', 'Login incorrect', 'Login timed out'] (timeout 00:48:53)"}
- {"dt": "2019-01-23T19:10:42.423096", "lvl": "target", "msg": "root”}
This is what I have in ser2net if I boot the board manually
Mbed Linux OS mbl-os-0.5.120 mbed-linux-os-8881 /dev/ttymxc0
mbed-linux-os-8881 login: root root@mbed-linux-os-8881-second-stage:~$
Which looks OK to me.
Any idea why the second prompt matching is not working?
Cheers
- If yes, does LAVA support multiple reboots?
Specify the actions in the test job actions list:
deploy, boot, test, boot, test
I’ve just tried but after the execution of the test, when it goes in to boot again, it redeploys the board again. I just want a simple reboot of the board.
- If yes, how can I write tests in order to run different sets of tests at any boot.
- Example: 1) do an upgrade 2) reboot the device 3) Check if the upgrade was successful
Specify a boot action to control how the device gets rebooted.
That’s a good point. I need to reboot WaRP7 and a RPi3 boards. And if I replicate the boot section it will deploy again the boardd. How can I skip the redeployment
- How can I structure my pipeline?
The hikey 6220 health check deploys two different systems, rebooting appropriately: https://staging.validation.linaro.org/scheduler/job/247930/definition
Also https://staging.validation.linaro.org/scheduler/job/246536/definition
Thanks
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
-- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
lava-users@lists.lavasoftware.org
-
Diego Russo -
Neil Williams