Hello everyone,
There seems to be a bug in LAVA. I was on version 2022.04 and have also tried 2023.03. Both versions have the same bug.
The same configurations works in a 2018 build of LAVA on an old machine.
I am trying to connect to an always on board via ssh.
The healthcheck is failing with this error :
lava-dispatcher, installed at version: 2023.03
start: 0 validate
Start time: 2023-04-12 14:07:00.373707+00:00 (UTC)
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 198, in validate self._validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 183, in _validate self.pipeline.validate_actions() File "/usr/lib/python3/dist-packages/lava_dispatcher/action.py", line 190, in validate_actions action.validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/actions/deploy/ssh.py", line 81, in validate if "serial" not in self.job.device["actions"]["deploy"]["connections"]: KeyError: 'connections'
validate duration: 0.00
case: validate
case_id: 244238
definition: lava
result: fail
Cleaning after the job
Root tmp directory removed at /var/lib/lava/dispatcher/tmp/8857
LAVABug: This is probably a bug in LAVA, please report it.
case: job
case_id: 244239
definition: lava
error_msg: 'connections'
error_type: Bug
result: fail
The health check looks like this:
job_name: SSH check
timeouts:
job:
minutes: 10
action:
minutes: 2
priority: medium
visibility: public
actions:
- deploy:
timeout: # timeout for the connection attempt
seconds: 30
to: ssh
os: oe
- boot:
timeout:
minutes: 2
prompts: ['root@(.*):~#']
method: ssh
connection: ssh
- test:
timeout:
minutes: 5
definitions:
- repository:
metadata:
format: Lava-Test Test Definition 1.0
name: smoke-tests-basic
description: "Basic smoke test"
run:
steps:
- lava-test-case linux-linaro-ubuntu-pwd --shell pwd
- lava-test-case linux-linaro-ubuntu-uname --shell uname -a
- lava-test-case linux-linaro-ubuntu-vmstat --shell vmstat
- lava-test-case linux-linaro-ubuntu-ip --shell ip a
from: inline
name: smoke-tests-basic
Any ideas ?
Best regards,
Sebastian
Hello everyone,
There seems to be a bug in LAVA. I was on version 2022.04 and have also tried 2023.03. Both versions have the same bug.
The same configurations works in a 2018 build of LAVA on an old machine.
I am trying to connect to an always on board via ssh.
The healthcheck is failing with this error :
lava-dispatcher, installed at version: 2023.03<https://10.1.52.17/scheduler/job/8857#L1>start: 0 validate<https://10.1.52.17/scheduler/job/8857#L2>Start time: 2023-04-12 14:07:00.373707+00:00 (UTC)<https://10.1.52.17/scheduler/job/8857#L3>Traceback (most recent call last): File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 198, in validate self._validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/job.py", line 183, in _validate self.pipeline.validate_actions() File "/usr/lib/python3/dist-packages/lava_dispatcher/action.py", line 190, in validate_actions action.validate() File "/usr/lib/python3/dist-packages/lava_dispatcher/actions/deploy/ssh.py", line 81, in validate if "serial" not in self.job.device["actions"]["deploy"]["connections"]: KeyError: 'connections'<https://10.1.52.17/scheduler/job/8857#L4>validate duration: 0.00<https://10.1.52.17/scheduler/job/8857#results_244238>case: validate
case_id: 244238
definition: lava
result: fail
<https://10.1.52.17/results/testcase/244238><https://10.1.52.17/scheduler/job/8857#L6>Cleaning after the job<https://10.1.52.17/scheduler/job/8857#L7>Root tmp directory removed at /var/lib/lava/dispatcher/tmp/8857<https://10.1.52.17/scheduler/job/8857#L8>LAVABug: This is probably a bug in LAVA, please report it.<https://10.1.52.17/scheduler/job/8857#results_244239>case: job
case_id: 244239
definition: lava
error_msg: 'connections'
error_type: Bug
result: fail<https://10.1.52.17/results/testcase/244239>
The health check looks like this:
job_name: SSH check
timeouts:
job:
minutes: 10
action:
minutes: 2
priority: medium
visibility: public
actions:
- deploy:
timeout: # timeout for the connection attempt
seconds: 30
to: ssh
os: oe
- boot:
timeout:
minutes: 2
prompts: ['root@(.*):~#']
method: ssh
connection: ssh
- test:
timeout:
minutes: 5
definitions:
- repository:
metadata:
format: Lava-Test Test Definition 1.0
name: smoke-tests-basic
description: "Basic smoke test"
run:
steps:
- lava-test-case linux-linaro-ubuntu-pwd --shell pwd
- lava-test-case linux-linaro-ubuntu-uname --shell uname -a
- lava-test-case linux-linaro-ubuntu-vmstat --shell vmstat
- lava-test-case linux-linaro-ubuntu-ip --shell ip a
from: inline
name: smoke-tests-basic
Any ideas ?
Best regards,
Sebastian
Hi Everyone,
My board has multiple uarts and they are designed to act as different
serial consoles of platform OS. So I am trying to set the desired
connection (serial) while calling the respective platform boot/flash
method from the job description. Means i need to utilise the same board and
have to call suitable serial console from connection list to get boot
logs.
{% set UART_PORTS = {"SE-UART": "AB0KOQF7", "UART2": "A10LOBA4", "UART4":
"AB0LPSO7"} %}
{% set connection_list = ['uart2', 'uart4'] %}
{% set connection_tags = {'uart2': ['primary','telnet'], 'uart4':
['telnet']} %}
{% set connection_commands = {'uart2': 'telnet 10.10.4.140 4004', 'uart4':
'telnet 10.10.4.140 4002'} %}
How to set required serial(uart2 or uart4) from connection list in job
description??
Regards
Nagendra S
You may want to have a look for Antonio's MR: https://git.lavasoftware.org/lava/lava/-/merge_requests/2038, not sure if it's your case.
-----Original Message-----
From: lava-users-request(a)lists.lavasoftware.org <lava-users-request(a)lists.lavasoftware.org>
Sent: Friday, March 24, 2023 8:00 AM
To: lava-users(a)lists.lavasoftware.org
Subject: [EXT] Lava-users Digest, Vol 54, Issue 8
Caution: EXT Email
Send Lava-users mailing list submissions to
lava-users(a)lists.lavasoftware.org
To subscribe or unsubscribe via email, send a message with subject or body 'help' to
lava-users-request(a)lists.lavasoftware.org
You can reach the person managing the list at
lava-users-owner(a)lists.lavasoftware.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Lava-users digest..."
Today's Topics:
1. Re: Not able to unpack the overlay other than "/" directory
(P T, Sarath)
----------------------------------------------------------------------
Message: 1
Date: Thu, 23 Mar 2023 07:14:49 +0000
From: "P T, Sarath" <Sarath_PT(a)mentor.com>
Subject: [Lava-users] Re: Not able to unpack the overlay other than
"/" directory
To: Milosz Wasilewski <milosz.wasilewski(a)foundries.io>
Cc: "lava-users(a)lists.lavasoftware.org"
<lava-users(a)lists.lavasoftware.org>, "Bhola, Bikram (PLM)"
<bikram.bhola(a)siemens.com>
Message-ID: <035b0a2cc7dd40f19552ce319acae2d9(a)mentor.com>
Content-Type: text/plain; charset="utf-8"
Hi Milosz,
Any update on below query ?
I changed "lava_test_results_dir": "/etc/lava-%s", from "lava_test_results_dir": "/lava-%s", from the above code session. Somehow the lava_test_results_dir variable which we defined in the job definition is not getting override by the default value , that’s why we did the change. Can you post your code snippet from the worker ? and do you know why the job definition is not able to identify the lava_test_results_dir value ?
Regards,
Sarath PT
-----Original Message-----
From: P T, Sarath
Sent: 15 March 2023 19:21
To: 'Milosz Wasilewski' <milosz.wasilewski(a)foundries.io>
Cc: lava-users(a)lists.lavasoftware.org; Bhola, Bikram (PLM) <bikram.bhola(a)siemens.com>
Subject: RE: [Lava-users] Re: Not able to unpack the overlay other than "/" directory
Hi Milosz,
Thanks for the support you have given. And I could able to achieve the scenario by changing the code from worker side as below
Navigate to the directory : cd /usr/lib/python3/dist-packages/lava_dispatcher
debian = {
"TESTER_PS1": r"linaro-test [rc=$(echo \$?)]# ",
"TESTER_PS1_PATTERN": r"linaro-test \[rc=(\d+)\]# ",
"TESTER_PS1_INCLUDES_RC": True,
"boot_cmds": "boot_cmds",
"line_separator": "\n",
# for lava-test-shell
"distro": "debian",
"tar_flags": "--warning no-timestamp",
"lava_test_sh_cmd": "/bin/bash",
"lava_test_dir": "/lava-%s",
"lava_test_results_part_attr": "root_part",
"lava_test_results_dir": "/etc/lava-%s",
"lava_test_shell_file": "~/.bashrc", }
I changed "lava_test_results_dir": "/etc/lava-%s", from "lava_test_results_dir": "/lava-%s", from the above code session. Somehow the lava_test_results_dir variable which we defined in the job definition is not getting override by the default value , that’s why we did the change. Can you post your code snippet from the worker ? and do you know why the job definition is not able to identify the lava_test_results_dir value ?
Regards,
Sarath P T
-----Original Message-----
From: Milosz Wasilewski <milosz.wasilewski(a)foundries.io>
Sent: 13 March 2023 14:50
To: P T, Sarath <Sarath_PT(a)mentor.com>
Cc: lava-users(a)lists.lavasoftware.org; Bhola, Bikram (PLM) <bikram.bhola(a)siemens.com>
Subject: Re: [Lava-users] Re: Not able to unpack the overlay other than "/" directory
Sarath,
Sorry, I can't reproduce any of your issues. You need to debug yourself.
Best Regards,
Milosz
On Mon, Mar 13, 2023 at 6:11 AM P T, Sarath <Sarath_PT(a)mentor.com> wrote:
>
> Hi Milosz,
>
> Any update ?
>
> Regards,
> Sarath P T
>
> -----Original Message-----
> From: P T, Sarath
> Sent: 08 March 2023 15:53
> To: 'Milosz Wasilewski' <milosz.wasilewski(a)foundries.io>
> Cc: lava-users(a)lists.lavasoftware.org; Bhola, Bikram (PLM)
> <bikram.bhola(a)siemens.com>
> Subject: RE: [Lava-users] Re: Not able to unpack the overlay other
> than "/" directory
>
> Hi Milosz,
>
> Now we could able to unpack the overlay in the path that we have given in the job definition. But after exporting the shell (export SHELL=/bin/bash) its not able to proceed with ". /<path_defined_in_definition>/lava-13030/environment" and "/<path_defined_in_definition>/lava-13030/bin/lava-test-runner /<path_defined_in_definition>/lava-13030/0 ". Below I am giving the job definition.
>
>
> priority: high
> visibility: public
> device_type: industrial-mtda-simatic-ipc227e-01
> visibility: public
> timeouts:
> job:
> minutes: 30
> action:
> minutes: 20
> connection:
> minutes: 20
> job_name: SLLL_CI_MTDA_SIMATIC_IPC227E_DEPLOY_BOOT_TEST_JOB
> actions:
> - deploy:
> to: overlay
> os: debian
> - boot:
> method: minimal
> reset: true
> failure_retry: 2
> auto_login:
> login_prompt: 'ebsy-isar login:'
> username: root
> password_prompt: "Password:"
> password: root
> prompts:
> - 'root@ebsy-isar:~#'
> transfer_overlay:
> download_command: sleep 10; cd /home/test ; wget
> unpack_command: tar -C /home/test -xzf
> - test:
> role:
> - node1
> timeout:
> minutes: 5
> definitions:
> - repository:
> metadata:
> description: basic test for some devices on board
> format: Lava-Test Test Definition 1.0
> name: kernel-version
> os:
> - debian
> run:
> steps:
> - cd /usr/libexec/ebsy-qa-suites/swupdate
> - ./swupdate_new.sh
> from: inline
> name: mtda-restart
> path: inline/mtda.yaml
> context:
> lava_test_results_dir: /home/test/lava-%s
> test_character_delay: 10
> tags:
> - slll-industrial-mtda-simatic-ipc227e-01
> timeouts:
> action:
> minutes: 20
> connection:
> minutes: 20
> job:
> hours: 7
> visibility: public
> job_name: SLLL_X86_TEST_JOB
> metadata:
> Description: '"Lava jobs for deploy, boot and usb test"'
> DEVICES: slll-simatic-ipc227e-mtda-01
>
> Regards,
> Sarath P T
>
>
>
> -----Original Message-----
> From: Milosz Wasilewski <milosz.wasilewski(a)foundries.io>
> Sent: 01 March 2023 14:27
> To: P T, Sarath <Sarath_PT(a)mentor.com>
> Cc: lava-users(a)lists.lavasoftware.org; Bhola, Bikram (PLM)
> <bikram.bhola(a)siemens.com>
> Subject: Re: [Lava-users] Re: Not able to unpack the overlay other
> than "/" directory
>
> On Wed, Mar 1, 2023 at 5:51 AM P T, Sarath <Sarath_PT(a)mentor.com> wrote:
> >
> > Hi Milosz,
> >
> > Yes it’s a strange behaviour that there is no command sent over serial to start overlay decompression. And is there any server configuration changes you made for accomplishing it ?
>
> No, I'm running vanilla LAVA dispatcher in a container from dockerhub
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhub.
> docker.com%2Fr%2Flavasoftware%2Flava-dispatcher%2F&data=05%7C01%7Clarr
> y.shen%40nxp.com%7C4bc23a099b864eb0dfbb08db2bfac2a1%7C686ea1d3bc2b4c6f
> a92cd99c5c301635%7C0%7C0%7C638152128229848153%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3
> 000%7C%7C%7C&sdata=D3wz%2BoY6viVxsjK%2FtFpRFBTeiftt%2B%2Bv6iB6E1oXwR%2
> BY%3D&reserved=0
>
> Best Regards,
> Milosz
------------------------------
End of Lava-users Digest, Vol 54, Issue 8
*****************************************
Hi everyone,
I'm experiencing this error on setting up my U-Boot based device:
Unable to extract cpio archive '/var/lib/lava/dispatcher/tmp/17/extract-overlay-ramdisk-tv17mriz/ramdisk.cpio': Command '['cpio', '--extract', '--make-directories', '--unconditional', '--file', '/var/lib/lava/dispatcher/tmp/17/extract-overlay-ramdisk-tv17mriz/ramdisk.cpio']' returned non-zero exit status 2.
As documentation in https://docs.lavasoftware.org/lava/actions-deploy.html#deploy-action-roles states, the overlay will only be used, when a test action is defined in job - so I removed the test action (and the log states "[common] skipped lava-overlay - no test action.") but the error still appears.
For debugging I commented out the _cleanup() function in /usr/lib/python3/dist-packages/lava_dispatcher/job.py (and can see in job output that dispatcher tmp directory isn't cleaned anymore) but somehow the directory still disappears, so I don't know how to debug this further.
So what's going wrong here? How to get the overlay working?
Running LAVA 2023.01 on Debian 11.6, full log see attached.
Thanks in advance!
Hi Team,
Facing an issue while unpack the overlay in some custom location other than "/" directory. From the LAVA linaro documentation I came to know that the -C / command to tar is essential or the test shell will not be able to start. Below are the issues that I am facing because of this.
* My / directory is a read only one and I want it in some /etc or /var.
* The LAVA job is getting stuck while doing the unpack with the command below in job definition.
transfer_overlay:
download_command: cd /tmp; wget
unpack_command: tar -C /tmp -xzf
Below I am attaching the test log from LAVA.
[cid:image001.png@01D9455A.1AD8F6A0]
How we can solve the read only filesystem image issue for unpacking the overlay other than "/" directory ?
Regards,
Sarath P T
Hey there,
I set up LAVA, and ran some jobs (e.g qemu from tutorial) which went fine - but I noticed HTTP 500 status code on /api/help page. Which wasn't severe as I thought I might ask that later. But this monday I discovered that /accounts/login/ isn't working anymore, HTTP 500 status code too! Of course I didn't change anything... (everybody says that) and honestly can't say when this broke as I only logged in once after install. Django log says "django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist."and as far as I can see Sites reside in database?
As I'm new to LAVA I don't know how to debug and to solve this, can you please help me? Is there a way to re-install the sites/templates?
Logs that might be relevant are attached.
After install (and qemu setup) I noticed that /var is full. This is resolved (deleted some docker images), but might be relevant.
Running on current Debian 11 as apt install, recent LAVA version.
Thanks in advance
Stefan
Hey everyone,
in the announcement of LAVA 2023.01 it says:
> The support for Debian Buster has been dropped as Debian Buster does
> not provide support for the latest pyyaml versions.
So I updated my LAVA machines from buster to bullseye.
Now I have the problem that I cannot install lava-dispatcher anymore now due to the following error:
The following packages have unmet dependencies:
libbpf0 : Depends: linux-libc-dev (>= 5.14) but 5.10.162-1 is to be installed
libbpf0 seems to come from the LAVA repository:
libbpf0:
Installed: (none)
Candidate: 1:0.5.0-1~bpo11+1~lava1
Version table:
1:0.5.0-1~bpo11+1~lava1 500
500 http://apt.lavasoftware.org/release bullseye/main amd64 Packages
1:0.3-2 500
500 http://ftp.de.debian.org/debian bullseye/main amd64 Packages
While linux-libc-dev is part of the standard debian repositories:
linux-libc-dev:
Installed: 5.10.162-1
Candidate: 5.10.162-1
Version table:
*** 5.10.162-1 500
500 http://security.debian.org/debian-security bullseye-security/main amd64 Packages
100 /var/lib/dpkg/status
5.10.158-2 500
500 http://ftp.de.debian.org/debian bullseye/main amd64 Packages
This is the LAVA repository I am using:
deb http://apt.lavasoftware.org/release bullseye main
Is this a known issue? How do I correctly install LAVA 2023.01 on Debian Bullseye?
Thanks in advance and kind regards,
Tim
--
Tim Jaacks
SOFTWARE DEVELOPER
SECO Northern Europe GmbH
Schlachthofstrasse 20
21079 Hamburg
Germany
T: +49 40 791899-183
E: tim.jaacks(a)seco.com
Register: Amtsgericht Hamburg, HRB 148893 Represented by: Dirk Finstel, Marc-Michael Braun, Massimo Mauri
Hello Lava Users,
On deploying the latest LAVA release(LAVA 2022.11.1) packages after building the packages on Debian11 Host , we are facing issue with deploying lava-dispatcher-host package on Debian 11 host(11.6).
On checking further we noticed https://git.lavasoftware.org/lava/lava/-/blob/master/debian/control , lava-dispatcher-host package depends on base-files (<< 11.1) but base-files on system is 11.1+deb11u6.
To support Deploying on latest Debian bullseye release should this be updated or are we missing something.
$ sudo dpkg -i lava-dispatcher-host_2022.11.1+11+bullseye_all.deb
(Reading database ... 283093 files and directories currently installed.)
Preparing to unpack lava-dispatcher-host_2022.11.1+11+bullseye_all.deb ...
Unpacking lava-dispatcher-host (2022.11.1+11+bullseye) over (2021.10+10+buster) ...
dpkg: dependency problems prevent configuration of lava-dispatcher-host:
lava-dispatcher-host depends on base-files (<< 11.1) | python3-bpfcc (>= 0.21); however:
Version of base-files on system is 11.1+deb11u6.
Package python3-bpfcc is not installed.
lava-dispatcher-host depends on base-files (<< 11.1) | linux-headers-amd64 | linux-headers-arm64 | linux-headers-generic; however:
Version of base-files on system is 11.1+deb11u6.
....
Thanks,
Hemanth.
Hi All,
I'm looking to automate overall testing process and post the test results
in to JIRA instead of going to Lava CI.
Is there any way that I can use robot framework to run the tests on Lava
and so robot framework produces results in XML file that can be used to
post into issue tracker.
Any suggestions would be helpful
Thanks,
Pavan