Hi folks,
The 2026.05 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# New debugfs implementation for generating mountable images
Instead of using `libguestfs` LAVA uses `debugfs` command from
`e2fsprogs`. This should in theory result in fewer bugs compared to
`libguestfs`.
To fallback to `libguestfs` implementation is in place if `debugfs` command
is not found.
An optional `overlay_backend` key in the deploy stage allows explicit
selection of the backend (`auto`, `e2fsprogs`, or `guestfs`):
```yaml
- deploy:
overlay_backend: e2fsprogs
images:
rootfs:
url: https://example.com/rootfs.ext4
format: ext4
overlays:
lava: true
```
# Device-types
## New device-types
* r9a09g057-rzv2h-evk
* imx952-15x15-evk
# Server
## Job cancellation tracking
Admin log entries are now recorded when a job is cancelled, making it
easier to track who cancelled a job and when.
## Database
New indexes on `state` and `health` columns improve performance when
filtering jobs. The deprecated `index_together` on `TestJob` has been
replaced with dedicated per-column indexes.
## 500 error responses
Server 500 errors now return JSON instead of HTML when the client sends
an `Accept: application/json` header (as `lava-worker` and `lava-run`
now do). This prevents raw HTML from appearing in job logs.
## UI improvements
Enable search for device-type and all workers pages. Also order tags by name
(case insensitive) in the devices page.
## Future proofing
A lot of changes has been done recently to remove deprecated and old django
and
python features from the LAVA source code. This was possible due to the
removal
of support for Debian 10.
Multiple fixes for Debian 13 are also part of this LAVA release:
* Fixed `text_offset` not being part of device type schema even thought it
was used by some u-boot devices.
* Fixed REST API pages having invalid layout on Debian 13.
* Fixed various deprecation warnings.
* Fixed LDAP settings not working on Debian 13.
* Fix OIDC settings not working on Debian 13.
# Dispatcher
## Docker test action
Docker test actions no longer require the boot action to stop at a
shell prompt. This allows booting minimal systems without a proper shell
and running tests exclusively from the Docker test shell on the worker.
## fastboot
### fastboot autodetect
When using fastboot auto-detect feature, LAVA will now run fastboot in a
container if that's requested in the job definition. Prio to this release,
LAVA
was ignoring the docker configuration for running the auto-detection step.
### DUT power sequence
LAVA is now downloading and preparing image prior to powering DUT on. This
aligns fastboot deployment with the typical download-then-deploy pattern
used
by the other deployment methods in LAVA.
## Test services
Environment variables from the device dictionary and job definition are
now written to a `.env` file in the test service repository. Docker
Compose reads this file automatically, making it easy to pass
per-device information to service containers.
Device dictionary example:
```jinja2
{% set environment = { 'BOARD_QDL_ID': 'D902AA38'} %}
```
Docker Compose example to expose the variable to the container:
```yaml
services:
srv1:
environment:
- BOARD_QDL_ID
```
## u-boot
`#conf@XXX` is no longer required for every device when using the
`bootm` command with FIT images. The config suffix is only appended
when `bootm_device_tree_config` is set in the device dictionary.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.04 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Security fixes
## Private job information leaked in device table
The device table was displaying the submitter and description of the
currently running job even when the user did not have permission to view
that job.
Now users without view permission will only see that the device is
"Running" a
job, without any job details.
# Breaking changes
## LXC support dropped
LXC support has been removed. It has been unmaintained and deprecated for
several years.
Users of LXC jobs must migrate to [Docker-based alternatives](
https://lava.readthedocs.io/en/latest/user/advanced-tutorials/running-arbit…
).
## Database migrations squashed (upgrade path limited)
`lava_results_app` migrations have been squashed to remove non-portable
PostgreSQL-specific SQL and Python. The squashed migration is now the
new initial migration.
**Direct upgrades from LAVA 2020.09 or earlier are no longer supported.**
The oldest supported upgrade path is from LAVA 2020.12 (the version
shipped with Debian 11 Bullseye). Administrators running older instances
must upgrade to at least 2020.12 before upgrading to this release.
## Worker auto-registration disabled by default
Worker auto-registration is now disabled by default due to security
implications.
If `WORKER_AUTO_REGISTER_NETMASK` is set and a HTTP proxy is in use, LAVA
sees
the proxy's IP address (possibly localhost), which could allow unauthorized
workers to register and obtain tokens.
To re-enable auto-registration, explicitly configure
`WORKER_AUTO_REGISTER_NETMASK`
in your settings.
## Debian 11 support
As of 2026.04 release Debian 11 (Bullseye) is no longer supported.
Administrators are requested to migrate to at least Debian 12 (Bookworm).
# LAVA Server
## REST API
New endpoints for managing groups and users are available in the v0.2 REST
API:
* `GET/POST /api/v0.2/groups/`
* `GET/PUT/PATCH/DELETE /api/v0.2/groups/<id>/`
* `GET/POST /api/v0.2/users/`
* `GET/PUT/PATCH/DELETE /api/v0.2/users/<id>/`
## XMLRPC API
Fix device owner not being updated when calling `lavacli devices update
--user`.
The `physical_owner` field was not being updated by the API call.
# LAVA Worker
## Test services
LAVA now supports running test services on the worker. Test services are
Docker Compose-based containers started on the LAVA worker, enabling custom
services for test definitions. For example, LAVA can now start an OTA server
that a DUT under test can connect to.
Job definition example:
```yaml
- test:
services:
- name: my-service
compose:
version: "3"
services:
ota-server:
image: my-ota-server:latest
ports:
- "8080:8080"
definitions:
- repository: https://example.com/tests.git
from: git
path: testdefs/ota.yaml
name: ota-test
```
## Expected test results
When an expected test list is provided, any test case not in that list now
results in a **fail** (previously it was only logged). This makes it easier
for regression-detection tools to spot unexpected test cases that should be
investigated.
## GRUB
## Overriding `net_commands`
Device dictionaries can now override `net_commands` to set custom GRUB
network
commands. This is useful for debugging network boot issues (e.g. adding
`net_ls_addr`):
```jinja2
{% set net_commands = ['net_ls_addr', 'dhcp'] %}
```
### Error detection
LAVA will now automatically catch a large set of error raised by Grub while
trying to boot a kernel.
The following error patterns are now matched:
* `missing (.*) symbol.`
* `couldn't send network packet`
* `destination unreachable`
* `couldn't autoconfigure`
* `timeout reading`
* `file '(.*)' not found`
* `no such device`
* `disk '(.*)' not found`
* `unknown filesystem`
## udev: kernel netlink fallback for containers
The udev monitoring code now falls back to kernel netlink if no udev
events are received. This allows containerized dispatchers to detect
USB devices without requiring `--network=host` or `CAP_NET_ADMIN`, since
containers don't propagate udev events across namespaces.
sysfs attributes are used as a fallback for `ID_SERIAL_SHORT` when udev
properties are unavailable.
## LAVA_JOB_ID environment variable
`LAVA_JOB_ID` is now available in the `lava-run` environment, accessible
to `user_commands` the same way as secrets declared in the job definition.
This is useful for running commands that are job-specific.
# Documentation
The documentation has been again vastly improved with changes to:
* Admin docs: hardware recommendations, typical deployment topologies,
LDAP configuration, first steps, recommended security settings
* Developer docs: contribution guide, code of conduct, guide for adding
new actions, new device type guide, developer first steps
* User docs: job debugging guide, job results, LAVA auth, user permissions,
common LAVA use cases
# CI
More pylint checks enabled and code brought into compliance:
`W1510`, `W0611`, `W1115`, `E0307`, `E0611`, `E1136`.
The minimum checked Python version has been increased to **3.11**.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.02 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Security fixes
Some security issues has been fixed in this release and we advice to
upgrade as soon as possible.
The details will be available later on.
# Device-types
## New device-types
* asus-CX3402CVA-brya
* lenovo-chrome-2in1-14iru10-brox
* mt8196-rauru-navi-sku1
* r9a07g043u11-smarc
* r9a07g044c2-smarc
* sc7180-trogdor-wormdingler-rev1-boe
# Documentation migration
Documentation v3 continues to be updated. All `deploy`, `boot` and `test`
methods are now documented and published at
https://lava.readthedocs.io/en/latest under **Technical references**.
# Debian support
Debian 11 (Bullseye) is being phased out. 2026.02 is the last LAVA release
to support it.
Starting from 2026.02 the oldest supported Debian release will be 12
(Bookworm)
# Docker
* All LAVA base images are now based on Debian 13 (Trixie) `slim`
* Debian 11 (Bullseye) was dropped from all CI jobs
* `wget` tool was added to lava-dispatcher-base image
# Job logs
## Secret filtering
LAVA is now filtering the job logs to remove sensitive data. This is still
a work in progress that will be improved in the following releases.
With this new feature, values of the variables defined in `secrets`
dictionary will be replaced in the job output log with string `[MASKED]`.
# Dispatcher changes
## Downloading artifacts
Set max_retries to 1 by default when downloading an artifacts.
The max_retries value can be set in the job definition using failure_retry
key. Example:
```yaml
- deploy:
failure_retry: 2
images:
image:
url: https://example.com/foo
```
## Deploy character delay
Add support for `deploy_character_delay` to add delay between each
character when interacting with the serial during deployment.
To use it add to the device dictionary (in milliseconds):
```jinja2
{% set deploy_character_delay = 30 %}
```
Defaults to 30ms delay for vexpress devices.
## fastboot
Improve fastboot device detection by making parsing more generic. This
should now work with multiple fastboot versions.
## FVP
Use `find` to locate armlm binary instead of hardcoded path to provide
compatibility with more FVP models.
## multinode
Fix `lava-role list` to properly match the documentation. Also fix
multinode collate function used to replace placeholdeers like `$ipaddr` in
synchronization messages.
## pyocd
Fix flashing with pyocd with latest pycocd versions.
## Transfer overlay
Allow to transfer the overlay using zmodem when network is not available.
In the job definition, add:
```yaml
- boot:
transfer_overlay:
transfer_method: zmodem
unpack_command: tar -C / -xzf
```
The worker and target device **must** have respectively the `sz` and `rz`
utilities installed. This is typically provided by: `lrzsz` package on
Debian-based or Fedora-based systems.
## uuu
Allow to use `uniquify` parameter for uuu deployment action. This is useful
when using `downloads://` urls.
# Server changes
## Restricting callback URLs
By default, notification callbacks can target any URL. Administrators can
restrict which hosts are permitted by setting `CALLBACK_ALLOWED_HOSTS` in
`/etc/lava-server/settings.yaml` or a file in
`/etc/lava-server/settings.d/`:
```yaml
CALLBACK_ALLOWED_HOSTS:
- "example.com"
- "*.example.com"
- "*.ci.example.com"
```
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.01 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* imx952-19x19-evk
* osm-imx93
* r8a779g3-sparrow-hawk
* juno-r2
* mt8188-geralt-ciri-sku4
* RZV2L
# LAVA Server
## LAVA monitoring
Two scripts for exporting test job data and device transitions as JSON was
added.
The scripts can be called with:
```shell
lava-server manage testjob_exporter --output-dir data/
lava-server manage transition_exporter --output-dir data/
```
The json files can then be consumed by [lava-dashboard](
https://gitlab.com/lava/dashboard/lava-dashboard) to generate graphs of the
device and job usages.
## UI improvements
Interminent worker to server communications errors used to be displayed at
the
top of the job details page. This is now collapsed by default for complete
jobs
and visible for incomplete jobs.
A link to the failing job is now available when the health of the device is
bad
(after a failed check for the device).
Improve test job table filtering
The following views are now properly checking properly user permissions::
* Plain device dictionary download
* Device health check list
* Worker details
## XMLRPC API
Allow to add a reason when updating the health. This was already available
in
the web interface and the REST API.
The reason of the latest health change is also shown on the device details
page.
# LAVA Worker
## Labgrid support
LAVA now supports running labgrid tests natively or within the LAVA Docker
test shell.
Native run example:
```shell
$ cd share/labgrid2lava/
$ python3 ./converter.py -l examples/uboot.yaml -o output/ -d
bcm2837-rpi-3-b-32
$ sudo lava-run --job-id 1 \
--output-dir ./1 \
--device ./output/device.yaml \
./output/job.yaml
```
Job definition example with LAVA Docker test shell:
```yaml
actions:
- test:
docker:
image: python:3.13
local: true
definitions:
- repository: https://github.com/Linaro/test-definitions.git
from: git
path: automated/linux/labgrid/labgrid.yaml
parameters:
UPDATE_ENV: true
LG_ENV: "./tests/example/env.yaml"
LG_TEST: "./tests/example/test_shell_strategy.py"
DEB_PKGS: "usbrelay"
name: labgrid-shell-strategy-example
```
## Misc
When manipulating cpio archives all operations are performed using
`fakeroot`.
This is fixing issues with devices nodes (like /dev/null) that where lost in
the process.
## Docker test shell
### Environment variables
Boot and test character delay are now available as environment variables
inside
the docker test shell: `LAVA_BOOT_CHARACTER_DELAYS` and
`LAVA_TEST_CHARACTER_DELAYS`.
### Serial disconnect
Test shell running in docker can now disconnect device connection to allow
test definition to use it. Test shell also has access to connection
specific environment variables.
Add this snippet to the job definition:
```yaml
- test:
docker:
image: debian:trixie
disconnect_connection: True
```
## FVP
Add support for running fvp shell commands. The `commands` are executed in
the
FVP Docker container after the FVP model has started. It allows running
custom
shell commands or a script that is already in the container to communicate
with
the model.
## usbg-ms deploy action
Allow users to run `usbg_ms_commands.disable` command in the job
definition, to disable usbg-ms whenever needed.
In order to call this function, add to the job definition:
```yaml
- command:
name: usbg_ms_commands_disable
```
## Boot action
### boot to bootloader
By default LAVA will reset the previous connection and the board power when
executing
the boot action. Users can skip this step by adding `reset_connection:
false` and `reset_device: false` respectively.
This version allows to skip reset with the boot to bootloader action.
### Transfer overlay
LAVA will now check that the overlay is properly downloaded and deployed
when using `transfer_overlay`. In case of failure, LAVA will fail the job.
## List of expected tests
List of expected tests is now supported in test definitions or job
definitions. Missing test cases are reported as fail, while extra test
cases are logged as warnings. These cases are always checked and reported
either after the test finishes or right before the job exists on errors.
All three test actions (definitions, monitors and interactive) support
providing the list in the same way.
Test definition example:
```yaml
metadata:
format: Lava-Test Test Definition 1.0
name: expected-testdef-example
run:
steps: []
expected:
- tc1
- tc2
- tc3
- tc4
```
Job definition example:
```yaml
- test:
definitions:
- repository: https://gitlab.com/lava/functional-tests.git
from: git
path: functional/expected-tests.yaml
name: expected-tests
expected:
- tc1
- tc2
```
## Fixed SShSession action raising exception
It became broken in 2025.10 release.
## Fixed RunNodebooterContainer action raising exception
It was broken since 2023.06 release.
# CI
Enable more pylint checks and fix code that wasn't compliant before.
Enabled checks include: C0121, C0123, C0201, C0206, C0207, C2801, R1701,
R1718, R1719, R1721, R1728, R1729, R1731, R1733, R1734, R1735, W0101,
W0102, W0106, W0108, W0127, W0130, W0640, W0706, W0715, W3101 and W4701
More details on the meaning of the codes can be found in [pylint wiki](
http://pylint-messages.wikidot.com/all-codes)
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2025.10 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* acer-cp714-1hn-rex
* acer-cpe794-1n-rex
* imx8mp-frdm
* imx91-11x11-frdm-imx91s
* imx943-15x15-lpddr4-evk
* imx95-15x15-frdm
* k3-am625-verdin-wifi-mallow
* mt8196-rauru-navi-sku2
* rk3576-rock-4d
## Removed device-types
* aml-s905d3-cc: duplicates of meson-sm1-s905d3-libretech-cc
## u-boot
Support dynamic device tree update using fdt command. This feature allows
dynamic modifications to the loaded DTB.
For example, the device dictionary could include the following to enable
the ethernet at runtime on the mt8365-genio-350-evk:
```jinja
{% set uboot_fdt_extra_commands = ['fdt set /soc/ethernet status okay'] %}
```
# LAVA Server
## Database migration
A new partial index on TestCase table to improve performances when creating
the Job Errors page.
## gunicorn optimization
Use gunicorn's `--preload` mode to optimize memory usage.
The preload mode will load all the Python files before forking the workers.
This allows the Python code to be shared in memory between worker processes
resulting in less overall RAM being used.
## XMLRPC API
Add a new `proxy.scheduler.device_types.devices` endpoint to list the
devices of a given device-type. The function allows to filter by device
health.
# LAVA worker
## Kernel crashes
When a kernel panic happens a board can reboot faster than the end of the
kernel panic could be printed. This will result the job error becoming
`wait for prompt timed out` which is very misleading.
In this release, the error message will be `Kernel panic - not syncing`
even if matching the end of kernel panic fails.
## Log queue
The `lava-run` process is now using a [multiprocessing.Queue](
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue)
instead of a pipe to keep pending log lines in memory.
`multiprocessing.Queue` uses a unlimited queue as a internal buffer and a
separate thread to feed the pipe. It helps prevent job execution from being
blocked when logs are not consumed from the pipe in time
## HTTP cache rules
This release introduced rules and fallback when using the http caching
service option. In the dispatcher configuration, you can now specify 4
different parameters to configure the http caching service, which URL to
exclude and include and to fallback or not to the original URL if the
caching service failed.
```jinja
http_url_format_string: "https://cache.lavasoftware.org/api/v1/fetch?url=%s"
http_cache_include_rules: ["https://.*example.com"]
http_cache_exclude_rules: ["https://10.*"]
http_cache_fallback_origin_url: True
```
## Log parsing performances
Remove the [pexpect](https://pexpect.readthedocs.io/en/stable/) delays for
both reading and writing to the DUT serial console. This is improving log
parsing speed when a lot of interaction are happening.
## Worker and job cancellation during restart
When restarting lava-worker, the process waits for attached lava jobs to
finish cleanly. Due to a bug in previous version, canceling such jobs on
the LAVA server was not taken into account by the worker. Starting in the
release, lava-worker will correctly cancel lava jobs.
## Worker logs
Log time in lava-worker and lava-docker-worker log file into UTC. This has
been broken since 2018.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hello LAVA users and developers,
LAVA has been around and useful for a lot of people for a long time.
For some years now, the list of LAVA maintainers as been shrinking down to
2 or even 1 developer (payed by Linaro).
In order to review all the patches that the LAVA developers are pushing, we
are looking for developers that want to become maintainer.
The idea would be to delegate some part of the maintenance of LAVA
sub-systems to some developers.
If you are interested, please reach out to me in the lava-users ML,
privately or in the new LAVA discord server at https://discord.gg/4dVatqyw
Thanks for your attention.
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2025.06 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* eic7700-hifive-premier-p550
* gs101-oriole (Pixel 6)
* s32g399a-rdb3
* spacemit-k1-bananapi-f3
* sun55i-a527-cubie-a5e
# lava-worker
lava-worker is now waiting for a job to finalize before sending a second
signal to a canceling job. This will leave more time for a job to finalize
cleanly.
# lava-dispatcher actions
## Logs
Always save logs to a local file in yaml format.
The `logs.yaml` can be used for debugging or post-parsing in the same job
with a docker test shell. The log file is saved to dispatcher download
directory so that it can be retrieved via the http server using the url
`http://
${LAVA_DISPATCHER_IP}/tmp/${LAVA_DISPATCHER_PREFIX}${LAVA_JOB_ID}/logs.yaml`.
Test example:
```yaml
- test:
docker:
image: lavasoftware/lava-dispatcher:latest
definitions:
- from: inline
repository:
metadata:
format: Lava-Test Test Definition 1.0
name: Job log parsing
run:
steps:
- wget http://
${LAVA_DISPATCHER_IP}/tmp/${LAVA_DISPATCHER_PREFIX}${LAVA_JOB_ID}/logs.yaml
# Steps for additional job log parsing
- tail logs.yaml
name: job-log-parsing
path: inline/job-log-parsing.yaml
```
## Deploy
### Downloads
Add `LAVA_DISPATCHER_PREFIX` to the deploy to downloads shell environment.
### Flasher
Allow flasher deploy method to pass `uniquify` parameter for downloading
images to a flat directory.
```yaml
- deploy:
to: flasher
images:
image1:
url: https://example.com/image1
image2:
url: https://example.com/image2
uniquify: false
```
## Boot
### Bootloader custom commands
Allow to replace `LAVA_JOB_ID` and `LAVA_DISPATCHER_IP` for custom boot
commands.
### Depthcharge
Allow to override bot commands in the job definition:
```yaml
- boot:
method: depthcharge
commands:
- "exit"
```
## u-boot
Allow chain loading grub. As U-Boot nowadays has full UEFI support it is
very common that we chain load grub from U-Boot as UEFI application.
Enable to boot from u-boot into grub without resetting the board.
# lava-server
## Remote artifact tokens API
Add api to manage Remote artifact tokens via XML-RPC or REST.
## Required login
Require login in order to access to `/scheduler/reports` and
`/scheduler/joberrors` than can be long to generate.
With the current bots crawling for data, this can kill a LAVA instance.
## `lava-server manage device-types prune`
This command will delete all unused device types from the database.
To be considered unused the device type has to have no devices and no jobs
that requested it.
The extra unused device types slow down certain iterations like health
check scheduling.
You can call this command from the command line with:
```bash
lava-server manage device-types prune --dry-run
```
Rgds.
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 205.04 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* aml-s905d3-cc
* bcm2837-rpi-3-b-plus
* bcm2712-rpi-5-b
* mt8188-geralt-ciri-sku0
* qcs6490-rb3gen2
* rk3588-rock-5-itx
* supermicro-as-2015hr-tnr
## U-boot device-types
Allow adding extra boot methods
On system ready devices we usually boot from u-boot into grub. To be able
to interact with such a chainloaded bootloader, a device can now use the
extra_boot_methods variable.
Add the following to a device dict entry:
```jinja
{% set extra_boot_methods %}
grub:
parameters:
bootloader_prompt: {{ bootloader_prompt|default('grub>') }}
needs_interrupt: {{ grub_needs_interrupt|default(true) }}
grub-efi:
parameters:
bootloader_prompt: {{ bootloader_prompt|default('grub>') }}
needs_interrupt: {{ grub_needs_interrupt|default(true) }}
{% endset %}
```
# LAVA dispatcher
## Kernel message parsing
Look for u-boot start message to detect DUT reset and fail the job.
## Deploy to downloads
Save "downloads://*" file to unique path by default.
This change unifies the default behavior across the `http(s)://`, `file://`
and `downloads://` schemas and it provides the `uniquify` parameter for
overwriting.
## Postprocess action
Export `HTTP_CACHE`, `LAVA_JOB_ID` and `LAVA_DISPATCHER_IP` as environment
variables in the docker container runner post-processing code.
`HTTP_CACHE` is also exported to docker test actions.
## Slow serial
For device-types with slow or unreliable serial connections, a parameter
has been present in LAVA for a long time: `test_character_delays`. This is
forcing LAVA to sleep some milliseconds between each characters sent to the
serial console.
With this release, the lava-test-runner script will now sleep for the same
amount of time before sending each test signal. This should help mitigating
the impact of these serial connections.
## lava-test-shell
Add a new option to pass the output for a given test result, the result
will be the same as calling the command with the `--shell` argument.
The passed quoted arguments are now properly preserved by lava-test-shell.
Previously the arguments had to be quoted multiple times to prevent them
being split. Now the argument will be preserved quoted. For example, before
`lava-test-shell "some arg"` the `some arg` would be split to to two so the
`"'some arg'"` quoting was required. Now `"some arg"` will always be a
single argument. This was undocumented change in last LAVA 2024.02 release.
# LAVA worker
Fix frequent worker offline events. After 2024.09, frequent online ->
offline -> online records are observed on most workers. It is a regression
introduced by the commit be0f40cbf7c7e95fb03bd83507a84ccaf43d9baf.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 205.02 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* imx8dxl-evk-dl2
* imx8dxl-orangebox
* imx8mp-ipi-lec
* imx8mp-tqma8mpql-mba8mpxl
* imx91-11x11-frdm
* imx943-19x19-lpddr4-evk
* imx943-19x19-lpddr5-evk
* mimxrt1170-evk
* mimxrt1180-evk
* mt8365-genio-350-evk
* mt8370-genio-510-evk
* mt8395-genio-1200-evk
* mt8390-genio-700-evk
* mt8395-radxa-nio-12l
* mt8188-geralt-ciri-sku1
* mt8188-geralt-ciri-sku2
## AVH
Add support for firmware package. In the job definition, add:
```yaml
fw_package:
url: https://example.com/kronos-baremetal-psa-secure-storage-1.0.zip
storage_file: virtio_0
root_partition: 1
```
## fastboot
Add support for auto fastboot device detection. If only one fastboot device
is attached to a dispatcher, LAVA can now automatically discover the adb
and fastboot serial number. In the device dictionary, add:
```jinja
{% set fastboot_auto_detection = true %}
```
## FVP
Allow to pass custom environment variables to the FVP binary. In the job
definition add `environment: ["HELLO=WORLD"]`
## JLink
### Custom erase and reset commands
Allow to customize erase and reset commands. In the device-type template
add:
```jinja
{% set erase_command = "erase" %}
{% set reset_command = "r" %}
```
## K3
Use common templates for K3 platforms. This template provides `booti`
addresses and also set `uboot_ramdisk_format` to `raw` by default.
### Multi-core boards
Allow JLink to connect to multi-core boards (e.g. M33 or M7). In the
device-type template, a new parameter lists the supported core types. By
default, LAVA will connect to the first in the list.
```jinja
{% set supported_core_types = ['M33', 'M7'] %}
```
To select another core, specify it in the job definition:
```yaml
- boot:
method: jlink
coretype : "M7"
[...]
```
### QEMU
Add `mips64el`, `ppc64le` and `riscv32` to the list of supported
architectures.
## u-boot devices
Add support for raw initramfs images. For device-type with recent u-boot,
use raw ramdisk format by updating the device-type template:
```jinja
{% set uboot_ramdisk_format = uboot_ramdisk_format|default('raw') %}
```
# LAVA dispatcher
## Timeout and retries
When only deploy/boot/test block timeout is defined, the same timeout is
used for all the actions inside the block. When a RetryAction's child
action fails on timeout, it leaves no time for the next retry to run as the
parent timeout is respected too.
This change propagates timeout by levels and retries for retry actions. It
uses `parent_timeout // parent_retries` for child action timeout. The logic
is similar to setting a smaller named/child action timeout manually.
This change might require some adaptations to the timeout. For instance
downloads are automatically retried 3 times, so the download timeout will
be divided by 3 with this change, allowing for some retries.
## lava-dispatcher-host
Make `lava-dispatcher-host` cli startup 3 times faster. This is import as
this is used in udev rules.
# LAVA server
## Database configuration
Allow to set database `CONN_MAX_AGE` and `CONN_HEALTH_CHECKS` from
environment variables:
```yaml
LAVA_SETTINGS_DB_CONN_MAX_AGE=60
LAVA_SETTINGS_DB_CONN_HEALTH_CHECKS=true
```
Or inside `LAVA_JSON_SETTINGS` with:
```
DB_CONN_MAX_AGE=60
DB_CONN_HEALTH_CHECKS=true
```
## Notifications
Allow to send notification to device owner. In the notification block, add:
```yaml
to:
user: "{LAVA_DEVICE_OWNER}"
```
## Scheduler queue timeout
Allow to specify a default schedule queue timeout. In the configuration add:
```yaml
QUEUE_TIMEOUT_HOURS=5 # In hours
```
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2024.09 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Upgrading
## Database migrations
This release includes migrations for both lava_scheduler_app and
lava_results_app. The migrations add unique constraints to fix potential
(and rare) race conditions when receiving and parsing job logs.
In order to apply the migrations, the database should not contains TestJobs
with duplicated sub_id, TestSets, TestSuites or TestCases. Otherwise the
migrations will fail.
An helper script as been added as part of this release to **remove test
jobs with such duplicates**. The helper **should** be ran **before**
running the migrations.
```shell
lava-server manage check_uniqueness --yes
```
When using `--yes`, the helper script will automatically **remove** the
faulty jobs.
# Device-types
## New device-types
* imx8mp-debix-model-a
* imx91-9x9-qsb
* imx93-11x11-frdm
* imx95-15x15-ab2
* imx95-15x15-evk
* imx95-19x19-verdin
* k3-am62p5-sk
* qrb4210-rb2-uboot
* r8a779f4-s4sk
* r9a07g044l2-smarc
# Deploy actions
# Downloads
Make overlay tarball available in downloads. When running postprocess
action in "downloads" deployment it is sometimes necessary to use LAVA
overlay. Example use case is when running tests on the board without
networking. This release makes it possible to, for example, append LAVA
overlay to the ramdisk during postprocessing step.
```yaml
actions:
- deploy:
images:
image:
url: 'https://example.com/Image'
dtb:
url: 'https://example.com/device.dtb'
ramdisk:
url: 'https://example.com/ramdisk.gz'
timeout:
minutes: 5
to: downloads
postprocess:
docker:
image: mkbootimage:master
steps:
- mkdir lava_overlay
- tar xvf overlay*.tar.gz -C lava_overlay
- "(cd lava_overlay ; find * | cpio -o -H newc -R +0:+0 | gzip -9
>> ../ramdisk.gz)"
- mkbootimg --header_version 2 --kernel Image --dtb device.dtb
--cmdline "earlycon clk_ignore_unused pd_ignore_unused audit=0" --ramdisk
ramdisk.gz --output boot.img
```
## Fasboot
Allow fastboot devices to use minimal and u-boot boot methods. Some boards
like the Qualcomm robotics platforms can run U-Boot by chainloading with
fastboot. Add support for using the U-Boot boot method as a second step
after "fastboot boot".
## u-boot
### Device tree overlay
Add support for device tree overlay when deploying to u-boot with tftp. The
resource should be called `dtbo`.
```yaml
actions:
- deploy:
to: tftp
dtbo:
url: https://example.com/dtb-overlay
```
The support should be added to the u-boot device-type with:
```jinja
{% set enable_dtbo_support = 'true' %}
```
Also specify the right addresses with:
```jinja
{% set bootm_dtbo_addr = '0x93c00000' %}
{% set bootz_dtbo_addr = bootm_dtbo_addr %}
{% set dtb_base_resize = dtb_base_resize | default(1048576) %}
```
### Skip power reset
Allow to skip resetting the device for u-boot. Allow user to define a
fastboot boot action to load u-boot boot image, and then define a u-boot
boot action to run u-boot level boot steps.
```yaml
actions:
- boot:
method: fastboot
[...]
- boot:
method: u-boot
reset: false
[...]
```
## USB mass-storage
Add a new method to deploy to USB mass storage gadget. This feature
requires hardware support on the worker and is currently only available for
u-boot DUTs (but can be extended to other DUTs).
The device dictionary should include:
```jinja2
{% set usbg_ms_commands = {
"disable": ["gadget-ms", "off"],
"enable": ["gadget-ms", "on", "{IMAGE}"],
} %}
```
The job definition should look like:
```yaml
actions:
- deploy:
to: usbg-ms
image:
url: https://example.com/.../20231109_raspi_4_bookworm.img.xz
compression: zstd
format: ext4
partition: 1
overlays:
lava: true
```
# LAVA server
## Rest API
Add a new endpoint to cancel jobs in batch by POSTing to the endpoint the
list of job ids.
## Scheduler
When finishing a job, the scheduler will now schedule the next jobs way
faster. The process for reporting the end of a job has been improved,
reducing the time between two consecutive jobs.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
