Hi folks,
The 2022.02 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
New supported devices:
* seco-c61
## d05
Use NFS version 3 by default when loading the root file-system with NFS.
## dephtcharge
Allow to boot from emmc. Typical extra kernel arguments for booting a
Chrome OS image are `root=/dev/mmcblk0p3 cros_debug cros_secure`.
## hp-x360-14-G1-sona
Automatically retry on boot failures to workaround bootloader issues.
## UUU
Enable "on worker" uuu_usb_otg_path configuration instead of modifying
devices definitions located on the master.
Before this change, the LAVA device dictionary contained static uuu USB ID
as a string, like:
```jinja
{% set uuu_usb_otg_path = '1:324' %}
```
The dictionary can now contain a command that will actually return this
string.
```jinja
{% set uuu_usb_otg_path_command = ['board-control', 'imx8mq-evk-01',
'extra', 'uuu_otg_path', '--silent'] %}
```
# docker + adb,fastboot
We often observe race conditions where a USB device will disconnect between
the time when it gets added to the docker command line as a `--device=
option` and the time that the container actually starts.
Instead of running one-off containers, start the containers first, wait for
them to be up, map devices into them with lava-dispatcher-host, and then
run the command. This way, if the device gets disconnected it will be
re-shared with the container once it is enumerated again.
# Settings
The LAVA server settings can now be configured from environment variables.
Every variables that starts with `LAVA_SETTINGS_` will be added to the
django settings.
For complex settings, admins can use LAVA_JSON_SETTINGS which is expected
to be a dictionary as json, base64 encoded. For instance:
```python
import base64
import json
data = {
"WORKER_AUTO_REGISTER_NETMASK": ["::1"]
}
print(base64.b64encode(json.dumps(data).encode("utf-8")).decode("utf-8"))
```
In order to use it, add to the environment:
`LAVA_JSON_SETTINGS="eyJXT1JLRVJfQVVUT19SRUdJU1RFUl9ORVRNQVNLIjogWyI6OjEiXX0="`.
See [the documentation](
https://lava.readthedocs.io/en/latest/admin/basic-tutorials/instance/config…
).
# Container images
Fix LDAP support when using LDAPS and upgrade sentry-sdk to 1.5.5 in
lava-server images.
# Debian packaging
Ensure `lava-celery-worker`, `lava-dispatcher-host` and
`lava-docker-worker` are restarted on upgrades.
# Worker
## Logging interval
By default, lava-run will send the new logs to the server every second. For
large labs, this would generate a consequent load on the server. This
interval is now configurable when calling `lava-docker-worker` and
`lava-worker`.
Rgds
--
Rémi Duraffort
TuxArchitect
Linaro
Hi folks,
The 2022.01.2 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
This release is reverting the "Balance job scheduling between submitters"
feature. In fact on large instances with large queue, this feature is
making scheduling way slower.
The feature will be reworked and reintroduced in a later release when the
performance issue has been fixed.
Rgds
--
Rémi Duraffort
TuxArchitect
Linaro
Hi folks,
The 2022.01.1 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
New supported devices:
* acer-cb317-1h-c3z6-dedede
* dell-latitude-5400-4305U-sarien (split from dell-latitude-5400-sarien)
* fsl-ls1046a-frwy
* fsl-ls2088a-rdb
* fsl-lx2162a-qds
* hp-x360-12b-ca0500na-n4000-octopus
* lenovo-TPad-C13-Yoga-zork
* rk3399-rock-pi-4b
* seco-b68
*
## fsl-l*-rdb
Allow to boot from an alternate bank. In order to use this feature, set
`uboot_altbank` to `True` in the job context:
```yaml
context:
uboot_altbank: true
```
Admins should set `uboot_altbank_cmd` in the device-type template.
# lava-dispatcher-host
Add support for docker device sharing under cgroups v2.
Under cgroups v2, device access control is done with BPF programs only.
When docker creates a container, it already attaches a BPF program to that
container cgroup. lava-dispather-host replaces that BPF program with one of
its own, that allows the regular list of devices containers can usually
access (`/dev/null`, `/dev/zero`, ...), plus all the devices shared with
the container. Subsequent device sharing with the same container overrides
that BPF program with a new one
# Scheduler
## Balance job scheduling between submitters
This patch is intended to prevent submitter starvation when jobs have the
same priority and single submitter sends a large batch of jobs.
It is implemented by determining a number of already running jobs for each
submitter and using it to annotate the jobs submitted to the queue. This
allows reordering the job queue for picking a job from the submitter that
could potentially be starving for an available device. To ensure fairness
in edge cases (equal number of running jobs, single device of given device
type) jobs sharing the same set of attributes are randomised.
## Active workers
Only schedule jobs on active workers. When a worker is in maintenance, no
jobs will be scheduled on the attached devices.
# kexec
Add an option to download the artifacts from deployment action. The job
would look like:
```yaml
- deploy:
to: download
images:
kernel:
url: https://example.com/Image.bin
initrd:
url: https://example.com/rootfs.cpio.gz
compression: gz
- boot:
method: minimal
auto_login:
[...]
prompts:
- "root@localhost:~# "
- boot:
method: kexec
deploy: true
kernel: /boot/Image.bin
initrd: /boot/rootfs.cpio.gz
command: /usr/sbin/kexec
[...]
```
# Overlay
Add support for tar filesystem. This allow to add overlays to nfsrootfs for
the boards that use NFS.
Rgds
--
Rémi Duraffort
LAV A and Tux Architect
Linaro
Hi folks,
The 2021.11 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
New supported devices:
* asus-cx9400-volteer
* lenovo-hr330a-7x33cto1ww-emag
* sc7180-trogdor-lazor-limozeen
## kontron-sl28
Define `initrd_high` and `fdt_high` as these boards are 64 bits.
## grub devices
### bootp interface
It's now possible to set the `bootp` interface when net booting with grub.
Admins should add to device-dict:
```jinja
{% set net_interface = "efinet2" %}
```
### base_kernel_args
Allow to set `extra_kernel_args` in grub jobs. This was impossible in
previous version due to a bug in the jinja templates.
This can be set in the device dictionary or the job definition context.
### FVP
The support for reading feedback channels when running an FVP model as been
improved thanks to the work of Paul Sokolovsky.
With these changes LAVA is able to consume and record most of the feedback
channels data.
Job submitter can also configure the timeout on feedback channels reading
in the finalize action:
```yaml
timeouts:
connections:
read-feedback: 20
```
With a large timeout, LAVA will have more time to consume the remaining
data when finalizing a job.
## Minimal boot
Add support for `pre_os_command` and `pre_power_command` in minimal boot
action.
The job definition would look like:
```yaml
- boot
method: minimal
pre_power_command: true
```
These commands can be useful to activate or deactivate external hardware
before powering on the DUT.
## NBD
In order to boot DUT with NBD support, LAVA was using `xnbd-server`.
This project is not maintained anymore so LAVA is now using `nbd-server`
instead.
# Management command
Admins can now list jobs with `lava-server manage jobs list`.
To list health check jobs than where submitted in the last 10 days:
```shell
lava-server manage jobs list --submitter lava-health --newer-than 10d
```
To run lxc jobs, that where submitted in the la 10 days:
```shell
lava-server manage jobs list --newer-than 10d --lxc
```
# Performances
The performances of the following pages has been improved drastically:
* /results
* /scheduler/mydevices
* /scheduler/mydevicetypehealthhistory
Rendering these pages will generate way fewer SQL queries.
Rgds
--
Rémi Duraffort
LAVA and TuxArchitect
Linaro
Hi folks,
The 2021.10 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
New supported devices:
* arduino-nano-33-ble
* imx8mp-ab2
* meson-sm1-odroid-c4
* pc-k10n78
## mt8192-asurada
mt8192-asurada as been renamed mt8192-asurada-rev1 to match the device-tree
name.
## FVP
In order to fix some issues with the secondary consoles that should be read
at all time, LAVA is now using a specific wrapper called `lava-outerr`.
The wrapper script consumes the sub-process std(out|err) as fast as
possible, keep the data in a buffer and written to std(out|err) whenever
possible.
## nrf52-nitrogen
Allow to set connect_before_flash. The value is False for backward
compatibility. You can set it in the device dictionary:
```jinja
{% set connect_before_flash = true %}
```
# External authentication
Since [2021.03](2021.03#authentication), LAVA can delegate the
authentication to GitLab using the `django-allauth` module.
This release add the possibility to use any supported [provider](
https://django-allauth.readthedocs.io/en/latest/providers.html).
In order to use it, add this to the configuration:
```yaml
AUTH_SOCIALACCOUNT: "{'gitlab':{'GITLAB_URL': 'https://gitlab.example.com
'}}"
```
See `django-allauth` documentation for the right parameters.
For background compatibility, `AUTH_GITLAB_URL` and `AUTH_GITLAB_SCOPE` are
still supported.
# Interactive tests
When running tests with LAVA test shell, LAVA will regularly read the
connections from the non-active namespaces. In the logs, this is called
"feedbacks". This is important for some device-types like fvp models.
Interactive tests are now also regularly reading the non-actives
connections to fix some issues with the fvp device-type.
# kexec
The kexec boot method is now able to crash the kernel to test the kdump
feature.
The job definition will look like:
```yaml
boot:
method: kexec
kernel: /home/vmlinux
dtb: /home/dtb
initrd: /home/initrd
command: /sbin/kexec
on_panic: true
boot_message: "Starting crashdump kernel"
prompts:
- 'root@debian:~#'
```
LAVA will use `--load-panic` instead of `--load` and will call `echo c >
/proc/sysrq-trigger` to force a kernel crash.
# Overlays
It's now possible to add single files to a given root file system (or
initrd) with the overlays feature.
The job definition should look like:
```yaml
rootfs:
image_arg: "-initrd {rootfs}"
url: "https://example.com/rootfs.cpio.zst"
compression: zstd
format: cpio.newc
overlays:
onefile:
url: https://example.com/file
format: file
path: /hello-world
```
# qemu and docker
The qemu-system packages included in the lava-dispatcher docker image are
now coming from bullseye-backports.
Rgds
--
Rémi Duraffort
LAVA and Tux Architect
Linaro
Hi folks,
The 2021.09 tag has been pushed to master on git.lavasoftware.org
<http://git.lavasoftware.org>.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release <https://apt.lavasoftware.org/release>
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/ <https://hub.lavasoftware.org/>
and
https://hub.docker.com/u/lavasoftware
<https://hub.docker.com/u/lavasoftware>
Changes in this release
==================
# Upgrading
## Database migrations
No DB migrations in this release.
# Device-types
## New device-types
New supported devices:
* Cypress cy8ckit-064s0s2-4343w
* qrb5165-rb5
* mt8192-asurada
## NXP
Make lpcxpresso55s69 support flasher deploy and minimal boot method for
trusted
firmware M.
# LAVA dispatcher
## Overlay
When using the lava_test_results_dir the compress overlay used to add
the whole
path to the tarball. This resulted in qemu guestfs mount to have double
path of
the lava_test_results_dir as a LAVA overlay home directory. This issue
is now
fixed by having the tarball contain only to bottom level directory from the
lava_test_results_dir parameter.
# Docker
## LAVA docker worker
Support providing a custom dockerfile so that users can provide the
container
with additional package installations or custom scripts they might require.
# Bug fixes
* Show template error clearly when using sync_to_lava feature.
* download: add a 60s timeout to every http request
Thank you,
--
Stevan Radaković | LAVA Senior Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hi folks,
The 2021.08 tag has been pushed to master on git.lavasoftware.org
<http://git.lavasoftware.org>.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release <https://apt.lavasoftware.org/release>
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/ <https://hub.lavasoftware.org/>
and
https://hub.docker.com/u/lavasoftware
<https://hub.docker.com/u/lavasoftware>
Changes in this release
==================
# Upgrading
## Database migrations
Added queue_timeout field to TestJob model.
# Device-types
## New device-types
New supported devices:
* asus-C436FA-Flip-hatch
* dell-latitude-5400-sarien
* gemini-ssi1328
* i945gsex-qs
* imx8mp-ddr4-evk
* intel-ixp42x-welltech-epbx100
* jh7100-beaglev-starlight
* hp-x360-14-G1-sona
* sm8150-mtp
* sm8250-mtp
* sm8350-mtp
## apertis
New deployment type for apertis is now available. Apertis is a Debian
derivative with images that typically do not have bash and often the
rootfs is
mounted read-only as well.
## jacuzzi
Device type acer-mt8183-cp311-3h-jacuzzi renamed to
mt8183-kukui-jacuzzi-juniper-sku16 to fit the device tree name.
## depthcharge
base-depthcharge device type template now supports flasher method and also a
minimal boot.
## qemu
Add cpu and machine context options for i386 arch.
## i.mx
Default ```shutdown_message``` was changed to 'Restarting system'
## pyocd
Allow to set the pyocd flash tool command. Example:
```yaml
{% set pyocd_command = "pyocd" %}
```
Specifically, for nitrogen board, allow to customize flash and boot cmds by
introducing pyocd_options to pyocd parameters. Defaults remain the same as
before.
## uuu
New option is now supported, ```uuu_power_off_before_corrupt_boot_media```
which will power down the device before the power up.
# API
## REST API
Test definition validation is now part of the LAVA REST API v2.
# LAVA server
## Scheduler UI
Add maintenance column to scheduler index view.
# LAVA dispatcher
## lava_test_shell
```ip``` command is now supported in lava test shell as in some yocto
releases
there is no /sbin/ifconfig but rather only /sbin/ip.
# Docker
## test docker action
When waiting for docker shell prompt, check it didn't die if there's
some error
in docker command line.
Allow to add arbitrary docker options through the device type. Example:
```yaml
actions:
deploy:
...
boot:
methods:
pyocd:
...
test:
methods:
docker:
global_options:
- --debug
options:
- --privileged
-
--mount=type=bind,source=/dev/disk/by-id,destination=/dev/disk/by-id
-
--mount=type=bind,source=/dev/serial/by-id,destination=/dev/serial/by-id
- {{ docker_shell_extra_arguments }}
```
## LAVA docker
debian:bullseye-slim is now used as base image.
# LXC
## lxc-mocker
Added support to use overlay for LXC rootfs. In order to provide support of
running multiple jobs at the same time in LXC mocker use a overlay
filesystem
to create a unique rootfs per LXC. This functionality can be activated using
LXC_MOCKER_USE_OVERLAY env variable which is disabled by default.
# Bug fixes
* Fix the documentation for "django migrations" setup command.
* Fix environment warnings during debian package upgrade
* Fix file permissions for jh7100-beaglev-starlight.jinja2 and
stm32mp157c-lxa-mc1.jinja2
* docs: fix dead link for XMLRPC methods
* pycocd: fix crash if pre_os_command is not defined
* kernel messages: fix order of messages
* scheduler_app: always select related submitter of current_job
* qemu: Don't disable the vland block in qemu
* qemu: lava_test_results_dir is now used as a mountpoint for guestfs
* debian: depends on uglifyjs instead of node-uglify
* schema: fix docker.network_from type
* download: fix crash when content-length == 0
* lava_rest_app: lock the job when calling cancel inside a transaction
* XMLRPC: fix TOCTOU crash
* docker: set dokerfile version to 1.2
* Job error: Failed to erase old recovery image is now and
Infrastructure error
Best regards,
--
Stevan Radaković | LAVA Senior Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hi folks,
The 2021.05 tag has been pushed to master on git.lavasoftware.org
<http://git.lavasoftware.org>.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release <https://apt.lavasoftware.org/release>
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/ <https://hub.lavasoftware.org/>
and
https://hub.docker.com/u/lavasoftware
<https://hub.docker.com/u/lavasoftware>
Changes in this release
==================
# Device-types
## New device-types
New supported devices:
* n1sdp
* stm32l562e-dk
* avenger96
* BeagleBone Enhanced
* asus-C523NA-A20057-coral
* asus-C433TA-AJ0005-rammus
* hp-x360-12b-n4000-octopus
## xilinx-zcu102
Fixed broken device type template.
## lpcxpresso55s69
Add support for "pyocd" boot method.
## u-boot
Add more uboot error messages encoutered on kernelci labs.
# LAVA server
## docker
### psql config generation
When running every services in the same container, generate the
configuration.
### Set environment for qemu in docker
When using qemu in docker, environment are not passed to it.
For the moment, only QEMU_AUDIO_DRV is usefull for qemu, so we pass only
it for limiting the length of the command line.
# Bug fixes
* lava-server: Remove console logs from profile template.
* doc: reminder to add the site to Chosen sites
* postinst: lava-scheduler is again a lava service
* security: Support security fix in pyyaml 5.4
* debian: lava-lxc-mocker: don't provide lxc
* lava_rest_app: test_api: don't subclass pathlib.Path
Thank you,
--
Stevan Radaković | LAVA Senior Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Hi folks,
The 2021.04 tag has been pushed to master on git.lavasoftware.org
<http://git.lavasoftware.org>.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release <https://apt.lavasoftware.org/release>
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/ <https://hub.lavasoftware.org/>
and
https://hub.docker.com/u/lavasoftware
<https://hub.docker.com/u/lavasoftware>
Changes in this release
==================
# Device-types
## New device-types
New supported device:
* kontron-pitx-imx8m
## imx8
The base_high_limits attribute default value is now false.
## lpcxpresso55s69
Add support for "pyocd" boot method.
## docker
### image dependencies
Two new apache2 mods are now enabled in lava-server docker image:
* mod_rewrite
* mod_ssl
Enabling these two modules by default will make docker based https
deployment
and upgrade easier.
The new dependency has been added to the docker image:
python3-django-allauth.
This makes gitlab SSO possible in the docker deployment.
### package versions
The following packages are now installed from the buster backports and
will thus
have a higher installation version:
* qemu-system-arm
* qemu-system-mips
* qemu-system-misc
* qemu-system-ppc
* qemu-system-x86
## FVP
Instead of launching a long-running container to just get the model version,
LAVA now does a one-off docker run call. This avoids leaving that container
running in the background.
# LAVA server
## notifications
A custom query argument named ```dependency_query``` is added for
notification
criteria section. If specified, the notifications will only be sent if
all the
jobs from the query satisfy the criteria condition specified in criteria
status
field. Example code:
```yaml
notify:
criteria:
status: finished
dependency_query: testjob__description__icontains__somestring
callback:
url: https://git.lavasoftware.org/api/v4/projects/1232/jobs/11113/play
method: POST
```
## configuration
LAVA now makes it possible to set the database, secret key and
ALLOWED_HOSTS for
each environment based on environment variables. It is useful for other use
cases as well, such as LAVA deployments based on containers.
Variables that can be set via env variables:
* DATABASE_URL
* INSTANCE_NAME
* SECRET_KEY
* ALLOWED_HOSTS
## Worker token API
It is now possible to set the worker token via the lava-server manage
command.
## Debian packaging
Downgraded postgresql and apache2 to a Recommends for lava-server,
making LAVA
setup more flexible.
## Master/worker version mismatch
Added a new setting variable ALLOW_VERSION_MISMATCH to disable the version
check.
Note: users should be aware of potential side effects of version mismatch.
# LAVA dispatcher
## boot/pyocd
A new option is available in pyocd boot action, ```connect_before_flash```.
This will allow to connect to device before flashing and capture serial
output
appearing immediately after flashing.
## failure retry interval
By default, individual action would be retried after 1 second, but you
can now
specify ``failure_retry_interval`` to increase the interval between retries.
```yaml
- deploy:
failure_retry_interval: 10
```
## overlay
LAVA is now adding env variables (from env-dut.yaml) to the secrets file in
overlay.
# Bug fixes
* lava-server manage sync: show device type when devices come back from
retired
* dispatcher: download: catch stderr when decompression fails
* doc: authentication: add notes to GitLab auth integration
* testdef: do not crash if a test step is None
* tar test definition: fix compression support
* containers: fix duplicate mount point for container share
Thank you,
--
Stevan Radaković | LAVA Senior Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
