Hi folks,
The 2026.07 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Database migation
3 database migrations are applied by this release:
* lava_scheduler_app.0065_alter_devicetype_health_frequency_default
* lava_scheduler_app.0066_blacklist_to_jsonfield
* lava_scheduler_app.0067_testjob_requested_device_and_worker
These migrations should not create any downtime, as they can run while
the instance is running. However, if you are running a large LAVA
deployment, be aware that 0066 rewrites the notification table in place,
so it needs temporary disk space of about that table's size and the table
is unavailable while it runs. 0067 may briefly stall the scheduler. The
new columns themselves are added instantly, but the indexes that come
with them block writes to the job table until the migration commits.
Generally this is pretty quick, a few seconds even on millions of jobs.
# Device-types
## New device-types
* stm32wba65i-dk1
* imx952-frdm
## QEMU: overlay drive interface
The `qemu` and `kvm` templates were attaching the `lava-guest.qcow2` test
overlay to an IDE drive (`if=ide`), which only works on x86 machines. On
other
architectures QEMU rejected the drive, so only boot-only tests could run.
The interface is now selected from the architecture:
* `ide` is kept on x86 for backward compatibility
* `virtio` is used everywhere else (aarch64, riscv, ppc, ...).
# Breaking changes
## qdl: `overlay_path` removed
`overlay_path` became a no-op in the `qdl` deploy action after the
`CompressOverlay` rework, and has been removed from the deploy schema. Jobs
still setting it will now fail validation and must simply drop the key:
```yaml
- deploy:
to: qdl
rootfs_image: "disk-sdcard.img2"
qcomflash:
url: 'https://example.com/qcomflash.tar.gz'
apply-overlay: true
```
# Server
## Target a specific device or worker
A job can now request one exact device or restrict scheduling to the devices
attached to one worker. The device must belong to the requested
`device_type`,
and the worker must exist on the instance:
```yaml
device_type: qemu
device: qemu-01
worker: worker-01
```
When `device` is used the scheduler will only run the job on that device,
and
will not fall back to another device of the same type. Likewise `worker`
restricts the job to devices of the requested type attached to that worker.
If
both are given, the selected device must also be attached to the selected
worker.
For multinode jobs, `device` and `worker` are specified inside each
`lava-multinode` role. A role-level `device` constraint requires that role
to
use `count: 1`.
## Require login on specific paths
Allow requiring login for specific URL paths.
`REQUIRE_LOGIN` gates a whole instance, which is too broad for otherwise
public
instances. The query UI, for example, is open to anonymous users and can
trigger expensive database queries, making large instances easy to overload
with indiscriminate crawlers.
The new `REQUIRE_LOGIN_PATHS` setting takes a list of URL path prefixes,
relative to the mount point. Each prefix covers itself and everything below
it:
```yaml
REQUIRE_LOGIN_PATHS: ["results/query", "results/chart"]
```
Requests under a configured prefix get the same enforcement as
`REQUIRE_LOGIN`,
including the standard exemptions and API token passthrough; everything else
stays open. An invalid configuration raises `ImproperlyConfigured` at
startup.
## Custom default health-check frequency
Device-types were always created with a routine health-check frequency of 24
hours. Admins can now change that default for an instance:
```yaml
HEALTH_FREQUENCY_HOURS: 168
```
The value is validated when the application starts: non-integer or
non-positive values are rejected with `ImproperlyConfigured`.
## Secrets in multinode jobs
A top level `secrets` section was silently dropped when submitting a
multinode
job. It is now copied to every child job of the multinode job, like the
other
shared sections.
## Notification blacklist stored as JSON
`Notification.blacklist` moved from a PostgreSQL-only `ArrayField` to a
`JSONField` so that it works on every database backend. A data migration
converts the existing `varchar[]` column to `jsonb` in place, preserving the
stored values. The conversion only runs on PostgreSQL instances where the
column is still an array, and is a no-op on fresh installs and on SQLite.
## Bug fixes
* **Login redirects**: `LOGIN_URL` is now computed with the mount point
prefix,
and `LOGIN_PATH` was replaced by a `login_path()` classmethod reading the
current setting at call time. With `MOUNT_POINT=lava/`, the login page
exemption check was failing and redirects were landing outside the
instance.
* **Compressed logs**: gunicorn used the `sendfile` kernel call when the
file
object of a `FileResponse` exposes `fileno()`. For LZMA files this
returned
the underlying file descriptor, so responses contained truncated, still
compressed data. The LZMA `fileno()` is now overridden to raise
`io.UnsupportedOperation`.
* **Empty settings file**: `yaml.safe_load()` returns `None` for an empty or
comment-only file, and calling `.items()` on it crashed LAVA at startup.
An
empty `settings.conf`, `settings.yaml` or `settings.d/*.yaml` now simply
contributes no setting.
* **ConfigFile**: the parser has been reworked on top of `shlex` instead of
regexes, fixing a file handle leak, restoring key validation and reporting
line numbers starting at 1.
# Dispatcher
## user_commands: lists of commands
`do` and `undo` in `user_commands` accepted a single command only, unlike
`power_on_command` and `power_off_command`. Both keys now also accept a
list of
commands, run in the order they are listed:
```jinja
{% set user_commands = {'set_boot_to_qspi': {'do': ['webrelay --host
192.168.0.15 --port 1234 --relay 1 on',
'webrelay --host
192.168.0.15 --port 1234 --relay 2 on'],
'undo': ['webrelay --host
192.168.0.15 --port 1234 --relay 2 off',
'webrelay --host
192.168.0.15 --port 1234 --relay 1 off']}} %}
```
Empty commands (and lists containing an empty command) are rejected at
validation time instead of silently running nothing. The device schema has
been
aligned with the implementation.
## Devices always powered off
The power cleanup now runs even when `CLEANUP_TIMEOUT` has already been
used up
by the previous cleanup steps, so boards are no longer left powered on
after a
job.
## Faster ext4 overlay
Two improvements to the `debugfs` based overlay introduced in 2026.05:
* Only the parent directory is resolved for non-directory tar members, the
basename being appended afterwards. Directory components are cached while
each file leaf is unique, so every file previously cost two uncacheable
`debugfs` stat calls. Building a large overlay took ~40 seconds for ~11
seconds of actual writes.
* Partition copies use a large `dd` block size.
Behaviour change: a file whose destination is an existing symlink in the
image
now replaces it instead of writing through it, which matches `tar` and
`libguestfs` `tar_in` semantics.
## Bug fixes
* **vexpress**: most exceptions raised while deploying are now reported as
`InfrastructureError` rather than job failures.
* **Overlay environment**: a debug message is logged when
`LAVA_DEVICE_HOSTNAME` or `LAVA_DEVICE_TYPE` are not exported because the
device configuration lacks the `hostname` or `devicetype` field.
* **Device-type templates**: variables unused by any base template have been
removed.
# Packaging
## sysusers.d
The Debian packages now ship `sysusers.d` configuration files instead of
creating users from hand-written maintainer scripts. This declarative
configuration also allows `/usr`-only image based systems to be created, and
`/etc` to be recreated on boot after a factory reset. It does not add a hard
dependency on systemd: debhelper generates the dependencies so that
non-systemd
and non-Linux Debian builds keep working.
## SQLite for local development
`DATABASE_URL` is now supported in the development settings and the model no
longer uses PostgreSQL-specific fields, so a development instance can run on
SQLite. The uv based development workflow is documented.
# CI
## pre-commit
The CI now runs a `pre-commit` job in the analyze stage, and most of the
checks
have been moved to hooks: `ruff` (check and format), `codespell`,
`pyupgrade`,
`django-upgrade`, `bandit`, `pylint`, `mypy` and the standard file checks
(large files, merge conflicts, private keys, TOML and YAML syntax, ...). The
`pyrefly` and `ty` hooks are enabled experimentally.
The now redundant `codespell`, `pylint`, `ruff` and SAST/`semgrep` CI jobs
and
their runner scripts have been removed, and `black` and `isort` have been
dropped from the development dependencies in favour of `ruff`.
## Supply chain
An SBOM is generated and scanned for vulnerabilities in CI, and the
`uv.lock`
file has been refreshed (closes CVE-2026-32274).
## Type checking
`lava_dispatcher/action.py` is now type checked. This required using
`isinstance` checks in `Action.parsed_command`, waiting on the
`subprocess.Popen` object rather than on `PexpectPopenSpawn`, and using
`subprocess.run` in `Action.run_command`. The asymmetric `errors`
getter/setter of `Action` and `Protocol` has been replaced by an
`errors_add`
method, since type checkers require a getter and a setter to use the same
type.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.06 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* lenovo-500e-gen5-rev3-nissa
* imx937-frdm
* imx941-19x19-lpddr4-evk
* imx942-19x19-lpddr4-evk
* tegra234-3737-0000+3701-0000
* devices deriving from the new `base-qdl` template (including
qcs6490-rb3gen2, which now supports minimal, u-boot and fastboot boot on
top
of qdl flashing)
## Renamed and removed device-types
* `e850-96` renamed to `exynos850-e850-96` to follow the device tree name.
* `juno-r2` dropped in favour of `juno`. The `juno-r2` template was an empty
alias with no overrides, and a single `juno` device-type covers all Juno
revisions.
# Breaking changes
## VLANd support dropped
The VLANd code has been finally removed from LAVA. This feature has been
unmaintained for years and was no longer used by any active deployment.
# Dispatcher
## QDL deploy and boot
LAVA now supports [`qdl`](https://github.com/linux-msm/qdl/), the tool used
to
flash Qualcomm devices. The typical device life cycle is:
* **deploy to qdl**: download a tarball containing all artifacts and add the
LAVA overlay
* **boot to qdl**: flash the board using the tarball contents. This step
can be
repeated to flash the various storage types (spinor, nvme, ufs, emmc, ...)
* boot using another method depending on the software build (minimal,
u-boot,
fastboot, ...)
`boot-to-qdl` also detects the installed `qdl` version and enables options
accordingly (for example `--skipblock`, which is only available on v2.7 and
above). The minimal version required to work with modern boards is 2.0.
`qdl` is not yet packaged in Debian, so it must be available on the
dispatcher
or inside the container. It has been added to the suggested packages of
`lava-dispatcher`.
Example job flashing and booting a `qcs6490-rb3gen2`:
```yaml
device_type: qcs6490-rb3gen2
job_name: qcs6490 qdl boot
timeouts:
job:
minutes: 210
visibility: public
actions:
- deploy:
to: qdl
rootfs_image: "disk-sdcard.img2"
overlay_path: "/"
qcomflash:
url: 'https://example.com/qcomflash.tar.gz'
apply-overlay: true
timeout:
minutes: 200
- boot:
method: qdl
firehose_program: "prog_firehose_ddr.elf"
rawprogram: "rawprogram0.xml"
patch: "patch0.xml"
storage: "ufs"
path: "."
timeout:
minutes: 5
- boot:
method: minimal
prompts:
- root@debian
timeout:
minutes: 3
```
The device dictionary should include the right configuration to enter EDL
mode.
```jinja
{% set board_qdl_id = 'BA9B2FEB' %}
{% set qdl_enter_commands = ['/usr/local/bin/tac-api.py --serial DP05DD00
--command bootToEDL'] %}
```
## rclone download support
A new `rclone` download backend allows downloading images via
[rclone](https://rclone.org/), giving access to 70+ cloud storage backends
(S3, Google Drive, Azure, SFTP, ...) through a unified URL scheme:
```yaml
- deploy:
to: tmpfs
images:
rootfs:
url: rclone://s3remote/bucket/images/rootfs.img.gz
compression: gz
```
Configuration can be provided from environment variable or the secret block:
```yaml
secrets:
rclone_env:
RCLONE_CONFIG_S3REMOTE_TYPE: "s3"
RCLONE_CONFIG_S3REMOTE_PROVIDER: "AWS"
RCLONE_CONFIG_S3REMOTE_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
RCLONE_CONFIG_S3REMOTE_SECRET_ACCESS_KEY:
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
RCLONE_CONFIG_S3REMOTE_REGION: "us-east-1"
actions:
- deploy:
to: tmpfs
images:
rootfs:
url: rclone://s3remote/bucket/images/rootfs.img.gz
compression: gz
```
## Docker login support
LAVA is now able to pull docker image from private docker registry using
username and password credentials provided in the job definition.
To pull images from a password protected registry add a login section
with the registry domain name, username and password.
```yaml
- test:
docker:
image: example.com/my-adb-image
login:
registry: example.com
user: foobar
password: my_password
```
## Test definition tarballs: strip-components
Tarballs fetched via `from: url` (such as GitHub and GitLab archives) often
wrap
their contents in a top-level directory.
LAVA now support a new optional `strip-components` parameter (passed
through to
`tar --strip-components`), so the test definition path can be given
relative to
the repository root.
This can be used with:
```yaml
- test:
definitions:
- name: smoke-tests
from: url
repository:
https://github.com/Linaro/test-definitions/archive/2019.03.tar.gz
compression: gz
strip-components: 1
path: automated/linux/smoke/smoke.yaml
```
## Faster tarball extraction
Test definition tarballs are now unpacked using the `tar` binary instead of
the
pure-Python `tarfile` module. This is significantly faster and also allows
unpacking Zstd-compressed archives.
## flasher: skip image download
Add an option to the `flasher` deploy action to not download an image but to
make it available in the substitution.
```yaml
- deploy:
to: flasher
images:
direct_url_image:
url: https://example.com/image.img.xz
download: false
```
The image is not downloaded; the URL from the job is passed directly to the
substitution. This avoids extra downloads and uploads when the flasher
device
can fetch the image itself.
## Device metadata in the shell environment
LAVA test action now export the device hostname and device type as
environment variables:
* `LAVA_DEVICE_HOSTNAME`
* `LAVA_DEVICE_TYPE`
## fastboot: pre-os-command always run
`pre-os-command` is now always run on `boot-to-fastboot`, including when the
action uses a Docker container. This fixes booting boards (such as RB1)
using
fastboot with an NFS-mounted rootfs.
## Bug fixes
* **lava-test-runner**: use KMSG in the runner when the test definition
uses it,
so the `EXIT` signal cannot arrive before `ENDRUN` and cause a run to be
wrongly marked as failed.
* **Overlay path**: `CompressOverlay` now stores the overlay under the full
`lava_test_results_dir` instead of just its basename, fixing a path
mismatch
when the variable is overridden in the context.
* **QEMU overlay**: the guest overlay is now created as `ext2` again, so
DUTs
with older kernels can mount it (a recent `e2fsprogs` enables
`metadata_csum_seed` by default).
* **vexpress**: default `test_character_delay` to `boot_character_delay` so
tests are paced like boot and deploy, helping slow consoles (such as
Juno's).
# Server
## Notifications: remote artifact tokens
Notification callbacks can now resolve secret tokens using the
`RemoteArtifactsAuth` table in addition to the legacy XML-RPC `AuthToken`
table.
Previously only XML-RPC tokens could be used as callback secrets, which
required
administrative privileges to set. Regular users can now manage their own
GitLab/API tokens for job notifications without administrative intervention.
# Packaging
## PEP 621 project metadata
The build now uses PEP 621 compliant project metadata and `setuptools-scm`
for
automatic version derivation from git tags. Dependencies are organised by
component (dispatcher, server, dispatcher-host, coordinator) so only what is
needed can be installed, with a `full` extra pulling in everything:
```sh
uv sync --extra full --extra test
pip install lava[server]
```
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.05 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# New debugfs implementation for generating mountable images
Instead of using `libguestfs` LAVA uses `debugfs` command from
`e2fsprogs`. This should in theory result in fewer bugs compared to
`libguestfs`.
To fallback to `libguestfs` implementation is in place if `debugfs` command
is not found.
An optional `overlay_backend` key in the deploy stage allows explicit
selection of the backend (`auto`, `e2fsprogs`, or `guestfs`):
```yaml
- deploy:
overlay_backend: e2fsprogs
images:
rootfs:
url: https://example.com/rootfs.ext4
format: ext4
overlays:
lava: true
```
# Device-types
## New device-types
* r9a09g057-rzv2h-evk
* imx952-15x15-evk
# Server
## Job cancellation tracking
Admin log entries are now recorded when a job is cancelled, making it
easier to track who cancelled a job and when.
## Database
New indexes on `state` and `health` columns improve performance when
filtering jobs. The deprecated `index_together` on `TestJob` has been
replaced with dedicated per-column indexes.
## 500 error responses
Server 500 errors now return JSON instead of HTML when the client sends
an `Accept: application/json` header (as `lava-worker` and `lava-run`
now do). This prevents raw HTML from appearing in job logs.
## UI improvements
Enable search for device-type and all workers pages. Also order tags by name
(case insensitive) in the devices page.
## Future proofing
A lot of changes has been done recently to remove deprecated and old django
and
python features from the LAVA source code. This was possible due to the
removal
of support for Debian 10.
Multiple fixes for Debian 13 are also part of this LAVA release:
* Fixed `text_offset` not being part of device type schema even thought it
was used by some u-boot devices.
* Fixed REST API pages having invalid layout on Debian 13.
* Fixed various deprecation warnings.
* Fixed LDAP settings not working on Debian 13.
* Fix OIDC settings not working on Debian 13.
# Dispatcher
## Docker test action
Docker test actions no longer require the boot action to stop at a
shell prompt. This allows booting minimal systems without a proper shell
and running tests exclusively from the Docker test shell on the worker.
## fastboot
### fastboot autodetect
When using fastboot auto-detect feature, LAVA will now run fastboot in a
container if that's requested in the job definition. Prio to this release,
LAVA
was ignoring the docker configuration for running the auto-detection step.
### DUT power sequence
LAVA is now downloading and preparing image prior to powering DUT on. This
aligns fastboot deployment with the typical download-then-deploy pattern
used
by the other deployment methods in LAVA.
## Test services
Environment variables from the device dictionary and job definition are
now written to a `.env` file in the test service repository. Docker
Compose reads this file automatically, making it easy to pass
per-device information to service containers.
Device dictionary example:
```jinja2
{% set environment = { 'BOARD_QDL_ID': 'D902AA38'} %}
```
Docker Compose example to expose the variable to the container:
```yaml
services:
srv1:
environment:
- BOARD_QDL_ID
```
## u-boot
`#conf@XXX` is no longer required for every device when using the
`bootm` command with FIT images. The config suffix is only appended
when `bootm_device_tree_config` is set in the device dictionary.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.04 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Security fixes
## Private job information leaked in device table
The device table was displaying the submitter and description of the
currently running job even when the user did not have permission to view
that job.
Now users without view permission will only see that the device is
"Running" a
job, without any job details.
# Breaking changes
## LXC support dropped
LXC support has been removed. It has been unmaintained and deprecated for
several years.
Users of LXC jobs must migrate to [Docker-based alternatives](
https://lava.readthedocs.io/en/latest/user/advanced-tutorials/running-arbit…
).
## Database migrations squashed (upgrade path limited)
`lava_results_app` migrations have been squashed to remove non-portable
PostgreSQL-specific SQL and Python. The squashed migration is now the
new initial migration.
**Direct upgrades from LAVA 2020.09 or earlier are no longer supported.**
The oldest supported upgrade path is from LAVA 2020.12 (the version
shipped with Debian 11 Bullseye). Administrators running older instances
must upgrade to at least 2020.12 before upgrading to this release.
## Worker auto-registration disabled by default
Worker auto-registration is now disabled by default due to security
implications.
If `WORKER_AUTO_REGISTER_NETMASK` is set and a HTTP proxy is in use, LAVA
sees
the proxy's IP address (possibly localhost), which could allow unauthorized
workers to register and obtain tokens.
To re-enable auto-registration, explicitly configure
`WORKER_AUTO_REGISTER_NETMASK`
in your settings.
## Debian 11 support
As of 2026.04 release Debian 11 (Bullseye) is no longer supported.
Administrators are requested to migrate to at least Debian 12 (Bookworm).
# LAVA Server
## REST API
New endpoints for managing groups and users are available in the v0.2 REST
API:
* `GET/POST /api/v0.2/groups/`
* `GET/PUT/PATCH/DELETE /api/v0.2/groups/<id>/`
* `GET/POST /api/v0.2/users/`
* `GET/PUT/PATCH/DELETE /api/v0.2/users/<id>/`
## XMLRPC API
Fix device owner not being updated when calling `lavacli devices update
--user`.
The `physical_owner` field was not being updated by the API call.
# LAVA Worker
## Test services
LAVA now supports running test services on the worker. Test services are
Docker Compose-based containers started on the LAVA worker, enabling custom
services for test definitions. For example, LAVA can now start an OTA server
that a DUT under test can connect to.
Job definition example:
```yaml
- test:
services:
- name: my-service
compose:
version: "3"
services:
ota-server:
image: my-ota-server:latest
ports:
- "8080:8080"
definitions:
- repository: https://example.com/tests.git
from: git
path: testdefs/ota.yaml
name: ota-test
```
## Expected test results
When an expected test list is provided, any test case not in that list now
results in a **fail** (previously it was only logged). This makes it easier
for regression-detection tools to spot unexpected test cases that should be
investigated.
## GRUB
## Overriding `net_commands`
Device dictionaries can now override `net_commands` to set custom GRUB
network
commands. This is useful for debugging network boot issues (e.g. adding
`net_ls_addr`):
```jinja2
{% set net_commands = ['net_ls_addr', 'dhcp'] %}
```
### Error detection
LAVA will now automatically catch a large set of error raised by Grub while
trying to boot a kernel.
The following error patterns are now matched:
* `missing (.*) symbol.`
* `couldn't send network packet`
* `destination unreachable`
* `couldn't autoconfigure`
* `timeout reading`
* `file '(.*)' not found`
* `no such device`
* `disk '(.*)' not found`
* `unknown filesystem`
## udev: kernel netlink fallback for containers
The udev monitoring code now falls back to kernel netlink if no udev
events are received. This allows containerized dispatchers to detect
USB devices without requiring `--network=host` or `CAP_NET_ADMIN`, since
containers don't propagate udev events across namespaces.
sysfs attributes are used as a fallback for `ID_SERIAL_SHORT` when udev
properties are unavailable.
## LAVA_JOB_ID environment variable
`LAVA_JOB_ID` is now available in the `lava-run` environment, accessible
to `user_commands` the same way as secrets declared in the job definition.
This is useful for running commands that are job-specific.
# Documentation
The documentation has been again vastly improved with changes to:
* Admin docs: hardware recommendations, typical deployment topologies,
LDAP configuration, first steps, recommended security settings
* Developer docs: contribution guide, code of conduct, guide for adding
new actions, new device type guide, developer first steps
* User docs: job debugging guide, job results, LAVA auth, user permissions,
common LAVA use cases
# CI
More pylint checks enabled and code brought into compliance:
`W1510`, `W0611`, `W1115`, `E0307`, `E0611`, `E1136`.
The minimum checked Python version has been increased to **3.11**.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.02 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Security fixes
Some security issues has been fixed in this release and we advice to
upgrade as soon as possible.
The details will be available later on.
# Device-types
## New device-types
* asus-CX3402CVA-brya
* lenovo-chrome-2in1-14iru10-brox
* mt8196-rauru-navi-sku1
* r9a07g043u11-smarc
* r9a07g044c2-smarc
* sc7180-trogdor-wormdingler-rev1-boe
# Documentation migration
Documentation v3 continues to be updated. All `deploy`, `boot` and `test`
methods are now documented and published at
https://lava.readthedocs.io/en/latest under **Technical references**.
# Debian support
Debian 11 (Bullseye) is being phased out. 2026.02 is the last LAVA release
to support it.
Starting from 2026.02 the oldest supported Debian release will be 12
(Bookworm)
# Docker
* All LAVA base images are now based on Debian 13 (Trixie) `slim`
* Debian 11 (Bullseye) was dropped from all CI jobs
* `wget` tool was added to lava-dispatcher-base image
# Job logs
## Secret filtering
LAVA is now filtering the job logs to remove sensitive data. This is still
a work in progress that will be improved in the following releases.
With this new feature, values of the variables defined in `secrets`
dictionary will be replaced in the job output log with string `[MASKED]`.
# Dispatcher changes
## Downloading artifacts
Set max_retries to 1 by default when downloading an artifacts.
The max_retries value can be set in the job definition using failure_retry
key. Example:
```yaml
- deploy:
failure_retry: 2
images:
image:
url: https://example.com/foo
```
## Deploy character delay
Add support for `deploy_character_delay` to add delay between each
character when interacting with the serial during deployment.
To use it add to the device dictionary (in milliseconds):
```jinja2
{% set deploy_character_delay = 30 %}
```
Defaults to 30ms delay for vexpress devices.
## fastboot
Improve fastboot device detection by making parsing more generic. This
should now work with multiple fastboot versions.
## FVP
Use `find` to locate armlm binary instead of hardcoded path to provide
compatibility with more FVP models.
## multinode
Fix `lava-role list` to properly match the documentation. Also fix
multinode collate function used to replace placeholdeers like `$ipaddr` in
synchronization messages.
## pyocd
Fix flashing with pyocd with latest pycocd versions.
## Transfer overlay
Allow to transfer the overlay using zmodem when network is not available.
In the job definition, add:
```yaml
- boot:
transfer_overlay:
transfer_method: zmodem
unpack_command: tar -C / -xzf
```
The worker and target device **must** have respectively the `sz` and `rz`
utilities installed. This is typically provided by: `lrzsz` package on
Debian-based or Fedora-based systems.
## uuu
Allow to use `uniquify` parameter for uuu deployment action. This is useful
when using `downloads://` urls.
# Server changes
## Restricting callback URLs
By default, notification callbacks can target any URL. Administrators can
restrict which hosts are permitted by setting `CALLBACK_ALLOWED_HOSTS` in
`/etc/lava-server/settings.yaml` or a file in
`/etc/lava-server/settings.d/`:
```yaml
CALLBACK_ALLOWED_HOSTS:
- "example.com"
- "*.example.com"
- "*.ci.example.com"
```
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2026.01 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* imx952-19x19-evk
* osm-imx93
* r8a779g3-sparrow-hawk
* juno-r2
* mt8188-geralt-ciri-sku4
* RZV2L
# LAVA Server
## LAVA monitoring
Two scripts for exporting test job data and device transitions as JSON was
added.
The scripts can be called with:
```shell
lava-server manage testjob_exporter --output-dir data/
lava-server manage transition_exporter --output-dir data/
```
The json files can then be consumed by [lava-dashboard](
https://gitlab.com/lava/dashboard/lava-dashboard) to generate graphs of the
device and job usages.
## UI improvements
Interminent worker to server communications errors used to be displayed at
the
top of the job details page. This is now collapsed by default for complete
jobs
and visible for incomplete jobs.
A link to the failing job is now available when the health of the device is
bad
(after a failed check for the device).
Improve test job table filtering
The following views are now properly checking properly user permissions::
* Plain device dictionary download
* Device health check list
* Worker details
## XMLRPC API
Allow to add a reason when updating the health. This was already available
in
the web interface and the REST API.
The reason of the latest health change is also shown on the device details
page.
# LAVA Worker
## Labgrid support
LAVA now supports running labgrid tests natively or within the LAVA Docker
test shell.
Native run example:
```shell
$ cd share/labgrid2lava/
$ python3 ./converter.py -l examples/uboot.yaml -o output/ -d
bcm2837-rpi-3-b-32
$ sudo lava-run --job-id 1 \
--output-dir ./1 \
--device ./output/device.yaml \
./output/job.yaml
```
Job definition example with LAVA Docker test shell:
```yaml
actions:
- test:
docker:
image: python:3.13
local: true
definitions:
- repository: https://github.com/Linaro/test-definitions.git
from: git
path: automated/linux/labgrid/labgrid.yaml
parameters:
UPDATE_ENV: true
LG_ENV: "./tests/example/env.yaml"
LG_TEST: "./tests/example/test_shell_strategy.py"
DEB_PKGS: "usbrelay"
name: labgrid-shell-strategy-example
```
## Misc
When manipulating cpio archives all operations are performed using
`fakeroot`.
This is fixing issues with devices nodes (like /dev/null) that where lost in
the process.
## Docker test shell
### Environment variables
Boot and test character delay are now available as environment variables
inside
the docker test shell: `LAVA_BOOT_CHARACTER_DELAYS` and
`LAVA_TEST_CHARACTER_DELAYS`.
### Serial disconnect
Test shell running in docker can now disconnect device connection to allow
test definition to use it. Test shell also has access to connection
specific environment variables.
Add this snippet to the job definition:
```yaml
- test:
docker:
image: debian:trixie
disconnect_connection: True
```
## FVP
Add support for running fvp shell commands. The `commands` are executed in
the
FVP Docker container after the FVP model has started. It allows running
custom
shell commands or a script that is already in the container to communicate
with
the model.
## usbg-ms deploy action
Allow users to run `usbg_ms_commands.disable` command in the job
definition, to disable usbg-ms whenever needed.
In order to call this function, add to the job definition:
```yaml
- command:
name: usbg_ms_commands_disable
```
## Boot action
### boot to bootloader
By default LAVA will reset the previous connection and the board power when
executing
the boot action. Users can skip this step by adding `reset_connection:
false` and `reset_device: false` respectively.
This version allows to skip reset with the boot to bootloader action.
### Transfer overlay
LAVA will now check that the overlay is properly downloaded and deployed
when using `transfer_overlay`. In case of failure, LAVA will fail the job.
## List of expected tests
List of expected tests is now supported in test definitions or job
definitions. Missing test cases are reported as fail, while extra test
cases are logged as warnings. These cases are always checked and reported
either after the test finishes or right before the job exists on errors.
All three test actions (definitions, monitors and interactive) support
providing the list in the same way.
Test definition example:
```yaml
metadata:
format: Lava-Test Test Definition 1.0
name: expected-testdef-example
run:
steps: []
expected:
- tc1
- tc2
- tc3
- tc4
```
Job definition example:
```yaml
- test:
definitions:
- repository: https://gitlab.com/lava/functional-tests.git
from: git
path: functional/expected-tests.yaml
name: expected-tests
expected:
- tc1
- tc2
```
## Fixed SShSession action raising exception
It became broken in 2025.10 release.
## Fixed RunNodebooterContainer action raising exception
It was broken since 2023.06 release.
# CI
Enable more pylint checks and fix code that wasn't compliant before.
Enabled checks include: C0121, C0123, C0201, C0206, C0207, C2801, R1701,
R1718, R1719, R1721, R1728, R1729, R1731, R1733, R1734, R1735, W0101,
W0102, W0106, W0108, W0127, W0130, W0640, W0706, W0715, W3101 and W4701
More details on the meaning of the codes can be found in [pylint wiki](
http://pylint-messages.wikidot.com/all-codes)
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2025.10 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* acer-cp714-1hn-rex
* acer-cpe794-1n-rex
* imx8mp-frdm
* imx91-11x11-frdm-imx91s
* imx943-15x15-lpddr4-evk
* imx95-15x15-frdm
* k3-am625-verdin-wifi-mallow
* mt8196-rauru-navi-sku2
* rk3576-rock-4d
## Removed device-types
* aml-s905d3-cc: duplicates of meson-sm1-s905d3-libretech-cc
## u-boot
Support dynamic device tree update using fdt command. This feature allows
dynamic modifications to the loaded DTB.
For example, the device dictionary could include the following to enable
the ethernet at runtime on the mt8365-genio-350-evk:
```jinja
{% set uboot_fdt_extra_commands = ['fdt set /soc/ethernet status okay'] %}
```
# LAVA Server
## Database migration
A new partial index on TestCase table to improve performances when creating
the Job Errors page.
## gunicorn optimization
Use gunicorn's `--preload` mode to optimize memory usage.
The preload mode will load all the Python files before forking the workers.
This allows the Python code to be shared in memory between worker processes
resulting in less overall RAM being used.
## XMLRPC API
Add a new `proxy.scheduler.device_types.devices` endpoint to list the
devices of a given device-type. The function allows to filter by device
health.
# LAVA worker
## Kernel crashes
When a kernel panic happens a board can reboot faster than the end of the
kernel panic could be printed. This will result the job error becoming
`wait for prompt timed out` which is very misleading.
In this release, the error message will be `Kernel panic - not syncing`
even if matching the end of kernel panic fails.
## Log queue
The `lava-run` process is now using a [multiprocessing.Queue](
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue)
instead of a pipe to keep pending log lines in memory.
`multiprocessing.Queue` uses a unlimited queue as a internal buffer and a
separate thread to feed the pipe. It helps prevent job execution from being
blocked when logs are not consumed from the pipe in time
## HTTP cache rules
This release introduced rules and fallback when using the http caching
service option. In the dispatcher configuration, you can now specify 4
different parameters to configure the http caching service, which URL to
exclude and include and to fallback or not to the original URL if the
caching service failed.
```jinja
http_url_format_string: "https://cache.lavasoftware.org/api/v1/fetch?url=%s"
http_cache_include_rules: ["https://.*example.com"]
http_cache_exclude_rules: ["https://10.*"]
http_cache_fallback_origin_url: True
```
## Log parsing performances
Remove the [pexpect](https://pexpect.readthedocs.io/en/stable/) delays for
both reading and writing to the DUT serial console. This is improving log
parsing speed when a lot of interaction are happening.
## Worker and job cancellation during restart
When restarting lava-worker, the process waits for attached lava jobs to
finish cleanly. Due to a bug in previous version, canceling such jobs on
the LAVA server was not taken into account by the worker. Starting in the
release, lava-worker will correctly cancel lava jobs.
## Worker logs
Log time in lava-worker and lava-docker-worker log file into UTC. This has
been broken since 2018.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hello LAVA users and developers,
LAVA has been around and useful for a lot of people for a long time.
For some years now, the list of LAVA maintainers as been shrinking down to
2 or even 1 developer (payed by Linaro).
In order to review all the patches that the LAVA developers are pushing, we
are looking for developers that want to become maintainer.
The idea would be to delegate some part of the maintenance of LAVA
sub-systems to some developers.
If you are interested, please reach out to me in the lava-users ML,
privately or in the new LAVA discord server at https://discord.gg/4dVatqyw
Thanks for your attention.
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 2025.06 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* eic7700-hifive-premier-p550
* gs101-oriole (Pixel 6)
* s32g399a-rdb3
* spacemit-k1-bananapi-f3
* sun55i-a527-cubie-a5e
# lava-worker
lava-worker is now waiting for a job to finalize before sending a second
signal to a canceling job. This will leave more time for a job to finalize
cleanly.
# lava-dispatcher actions
## Logs
Always save logs to a local file in yaml format.
The `logs.yaml` can be used for debugging or post-parsing in the same job
with a docker test shell. The log file is saved to dispatcher download
directory so that it can be retrieved via the http server using the url
`http://
${LAVA_DISPATCHER_IP}/tmp/${LAVA_DISPATCHER_PREFIX}${LAVA_JOB_ID}/logs.yaml`.
Test example:
```yaml
- test:
docker:
image: lavasoftware/lava-dispatcher:latest
definitions:
- from: inline
repository:
metadata:
format: Lava-Test Test Definition 1.0
name: Job log parsing
run:
steps:
- wget http://
${LAVA_DISPATCHER_IP}/tmp/${LAVA_DISPATCHER_PREFIX}${LAVA_JOB_ID}/logs.yaml
# Steps for additional job log parsing
- tail logs.yaml
name: job-log-parsing
path: inline/job-log-parsing.yaml
```
## Deploy
### Downloads
Add `LAVA_DISPATCHER_PREFIX` to the deploy to downloads shell environment.
### Flasher
Allow flasher deploy method to pass `uniquify` parameter for downloading
images to a flat directory.
```yaml
- deploy:
to: flasher
images:
image1:
url: https://example.com/image1
image2:
url: https://example.com/image2
uniquify: false
```
## Boot
### Bootloader custom commands
Allow to replace `LAVA_JOB_ID` and `LAVA_DISPATCHER_IP` for custom boot
commands.
### Depthcharge
Allow to override bot commands in the job definition:
```yaml
- boot:
method: depthcharge
commands:
- "exit"
```
## u-boot
Allow chain loading grub. As U-Boot nowadays has full UEFI support it is
very common that we chain load grub from U-Boot as UEFI application.
Enable to boot from u-boot into grub without resetting the board.
# lava-server
## Remote artifact tokens API
Add api to manage Remote artifact tokens via XML-RPC or REST.
## Required login
Require login in order to access to `/scheduler/reports` and
`/scheduler/joberrors` than can be long to generate.
With the current bots crawling for data, this can kill a LAVA instance.
## `lava-server manage device-types prune`
This command will delete all unused device types from the database.
To be considered unused the device type has to have no devices and no jobs
that requested it.
The extra unused device types slow down certain iterations like health
check scheduling.
You can call this command from the command line with:
```bash
lava-server manage device-types prune --dry-run
```
Rgds.
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
Hi folks,
The 205.04 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://registry.gitlab.com/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
==================
# Device-types
## New device-types
* aml-s905d3-cc
* bcm2837-rpi-3-b-plus
* bcm2712-rpi-5-b
* mt8188-geralt-ciri-sku0
* qcs6490-rb3gen2
* rk3588-rock-5-itx
* supermicro-as-2015hr-tnr
## U-boot device-types
Allow adding extra boot methods
On system ready devices we usually boot from u-boot into grub. To be able
to interact with such a chainloaded bootloader, a device can now use the
extra_boot_methods variable.
Add the following to a device dict entry:
```jinja
{% set extra_boot_methods %}
grub:
parameters:
bootloader_prompt: {{ bootloader_prompt|default('grub>') }}
needs_interrupt: {{ grub_needs_interrupt|default(true) }}
grub-efi:
parameters:
bootloader_prompt: {{ bootloader_prompt|default('grub>') }}
needs_interrupt: {{ grub_needs_interrupt|default(true) }}
{% endset %}
```
# LAVA dispatcher
## Kernel message parsing
Look for u-boot start message to detect DUT reset and fail the job.
## Deploy to downloads
Save "downloads://*" file to unique path by default.
This change unifies the default behavior across the `http(s)://`, `file://`
and `downloads://` schemas and it provides the `uniquify` parameter for
overwriting.
## Postprocess action
Export `HTTP_CACHE`, `LAVA_JOB_ID` and `LAVA_DISPATCHER_IP` as environment
variables in the docker container runner post-processing code.
`HTTP_CACHE` is also exported to docker test actions.
## Slow serial
For device-types with slow or unreliable serial connections, a parameter
has been present in LAVA for a long time: `test_character_delays`. This is
forcing LAVA to sleep some milliseconds between each characters sent to the
serial console.
With this release, the lava-test-runner script will now sleep for the same
amount of time before sending each test signal. This should help mitigating
the impact of these serial connections.
## lava-test-shell
Add a new option to pass the output for a given test result, the result
will be the same as calling the command with the `--shell` argument.
The passed quoted arguments are now properly preserved by lava-test-shell.
Previously the arguments had to be quoted multiple times to prevent them
being split. Now the argument will be preserved quoted. For example, before
`lava-test-shell "some arg"` the `some arg` would be split to to two so the
`"'some arg'"` quoting was required. Now `"some arg"` will always be a
single argument. This was undocumented change in last LAVA 2024.02 release.
# LAVA worker
Fix frequent worker offline events. After 2024.09, frequent online ->
offline -> online records are observed on most workers. It is a regression
introduced by the commit be0f40cbf7c7e95fb03bd83507a84ccaf43d9baf.
Rgds
--
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro
