Upgrading
=========
Device-type templates
---------------------
Device-type templates as been moved from
`/etc/lava-server/dispatcher-config/device-types/` to
`/usr/share/lava-server/device-types/`.
Database migrations
-------------------
A new `version` field as been added to the `Worker` model. The field will be used to track remove dispatcher version and check for incompatibilities between master and dispatcher.
Device-types
============
New device-types
----------------
New supported devices:
* Fixed Virtual Platforms
* fsl-ls1043a-rdb
* fsl-lx2160a-rdb
* fsl-s32v234sbc
* imx6q-var-dt6customboard
* imx6q-sabresd
* imx7d-sdb
* imx8mm-ddr4-evk
* imx8mm-evk
* r8a7795-h3ulcb-kf
* sun8i-h3-bananapi-m2-plus
Fixed Virtual Platforms
-----------------------
FVP jobs can now be ran in LAVA. LAVA will execute FVP devices inside Docker
containers that should be provided by job submitter.
A job definition would be:
```yaml
- deploy:
to: fvp
images:
bl1:
url:
https://example.com/fvp/bl1.bin[...]
ramdisk:
url:
https://example.com/fvp/ramdisk.img- boot:
method: fvp
docker:
name: "foundation:11.9"
local: true
image: /opt/model/Foundation_Platformpkg/models/Linux64_GCC-6.4/Foundation_Platform
version_string: 'ARM V8 Foundation Platformr0p0 [^\\n]+'
timeout:
minutes: 7
console_string: 'terminal_0: Listening for serial connection on port (?P<PORT>\d+)'
arguments:
- "--cores=4"
- "--no-secure-memory"
- "--visualization"
- "--gicv3"
- "--data={BL1}@0x0"
- "--data={FIP}@0x8000000"
- "--data={IMAGE}@0x80080000"
- "--data={DTB}@0x82000000"
- "--data={RAMDISK}@0x84000000"
- "--block-device={ROOTFS}"
prompts:
- 'root@genericarmv8:~#'
```
To use this new device-type, the job definition should define a
[deploy](
https://docs.lavasoftware.org/lava/actions-deploy.html#to-fvp)
and a
[boot](
https://docs.lavasoftware.org/lava/actions-boot.html#fvp)
action.
More information in the [FVP documentation](
https://docs.lavasoftware.org/lava/fvp.html).
UBoot USB Mass Storage
----------------------
LAVA dispatcher will now use `bmaptool` instead of `dd` to flash images. The
layout is computed right before the flash.
Docker test action
==================
The docker test action introduced in LAVA
[2020.02](
https://git.lavasoftware.org/lava/lava/-/wikis/releases/2020.02/) has
been improved.
Serial connection to the device
-------------------------------
The connection commands are now exposed in the test environment as
`LAVA_CONNECTION_COMMAND` and `LAVA_CONNECTION_COMMAND_*`.
For example the given connection commands:
```jinja
{% set connection_list = ['uart0', 'uart1'] %}
{% set connection_commands = {'uart0': 'telnet localhost 4002', 'uart1': 'telnet 192.168.1.200 8001'} %}
```
Will be exported as:
```shell
LAVA_CONNECTION_COMMAND='telnet 192.168.1.200 8001'
LAVA_CONNECTION_COMMAND_UART0='telnet localhost 4002'
LAVA_CONNECTION_COMMAND_UART1='telnet 192.168.1.200 8001'
```
Waiting for device on USB made optional
---------------------------------------
By default, the docker test action will wait for the device to be connected and exposed to the host via its USB OTG port.
In some use cases, however, you need to e.g. interact with u-boot before a USB OTG port is enabled, so waiting on the device coming up on USB would not work. To avoid waiting for the USB connection, you can specify the `docker.wait.device` parameter set to false:
```yaml
actions:
# ....
- test:
docker:
image: my-image
wait:
device: false
# ....
```
Device type templates
=====================
The default device-type templates has been moved from
`/etc/lava-server/dispatcher-config/device-types` to
`/usr/share/lava-server/device-types/`.
When rendering device dictionaries, LAVA will use templates in
`/etc/lava-server/dispatcher-config/device-types` and then fallback to
`/usr/share/lava-server/device-types/`. This mechanism allow admins to override
device-type templates.
When using the APIs to override device-type templates, LAVA will write the
template into `/etc/lava-server/dispatcher-config/device-types`.
Compression formats
===================
[zstd](
https://facebook.github.io/zstd/) as been added to the list of supported compression formats.
In order to use it in a job definition you should use:
```yaml
actions:
- deploy:
rootfs:
url:
https://example.com/rootfs.ext4.zst compression: zstd
```
The `zstd` package should be installed on the dispatcher. This is already the case for the lava-dispatcher docker image.
Postprocessing images with docker
=================================
This release adds support for postprocessing downloaded images using user-provider docker containers. To make use of this feature, you need to use the new ***downloads*** deploy target. Example:
```yaml
actions:
- deploy:
to: downloads
images:
# ...
postprocess:
docker:
image: my-image
steps:
- /path/to/my/postprocessing/script
```
The provided docker image will be run with the download directory as current working directory, and the commands listed in `steps` will be executed. Any changes that the commands make to the downloaded images are persisted for later actions, and this includes not only modifying the existing images, but also creating new files in there.
To make use of any files left in the downloads directory, you need a second deploy action, which can refer to files in the downloads directory using the ***downloads://*** pseudo URL scheme. Example:
```yaml
- deploy:
to: fastboot
docker:
image: my-adb-fastboot-image
images:
ptable:
url: downloads://ptable-linux-8g.img
reboot: hard-reset
boot:
url: downloads://boot.img
reboot: hard-reset
system:
url: downloads://rpb-console-image-lkft.rootfs.img
apply-overlay: true
```
Interactive non-exiting commands
================================
Interactive test received a new field: ```wait_for_prompt```. It defaults to True but can be set to False in the job definition. Idea behind this feature is to allow for non exiting commands to work in non-posix shells. Example can be invoking fastboot in u-boot shell and flashing the board from the subsequent test action
```yaml
- test:
timeout:
minutes: 10
interactive:
- name: erase-emmc
prompts: ['=> ']
script:
- command: mmc dev 1 0
name: mmc_dev
successes:
- message: mmc1(part 0) is current device
- command: mmc rescan
name: mmc_rescan
- command: mmc erase 0 0x400000
name: mmc_erase
successes:
- message: "4194304 blocks erased: OK"
- name: fastboot
prompts: ['=> ', '/ # ']
script:
- command: env default -f -a
name: env-default
successes:
- message: Resetting to default environment
- command: setenv partitions $partitions_android
name: setenv-partitions
- command: fastboot 1
success:
- message: "\n"
wait_for_prompt: false
```
Boot QEMU from docker image
===========================
We now allow users to provide their own docker image that LAVA will use to
start QEMU.
This will allow users to use recent QEMU version and to test QEMU itself.
Job definition schema example:
```yaml
- boot:
method: qemu
timeout:
minutes: 2
media: tmpfs
docker:
image: my-qemu-image
binary: /usr/bin/qemu-system-x86_64
prompts:
- "root@debian:"
auto_login:
login_prompt: "login:"
username: root
```
API
===
New certificate endpoints
-------------------------
We've added a support for downloading master certificate and for uploading a
worker certificate to master. This will make setting up a remote worker a bit
easier for the administrators.
New endpoints available for REST API:
* ```/api/v0.2/workers/${hostname}/certificate/```
* ```/api/v0.2/system/certificate/```
The first one excepts both GET and POST requests so the user can also download
the certificate for worker for double checking, while the second one can be
used only via GET request and will download the master public key.
New methods available for XMLRPC API:
* #scheduler.workers.get_certificate($hostname)
* #scheduler.workers.set_certificate($hostname, $key)
* #system.get_master_certificate()
They mirror the REST API calls described above.
Authorization
=============
New permissions backend for workers
-----------------------------------
The worker model is now protected by the "new" authorization system. You can
find settings in the admin section for managing individual workers.
Only `change` permission is used for workers, in order to allow administrators
to control who can upload worker certificate, environment and
configuration files.
Note that the worker authorization settings will **NOT** affect the device or
device type authorization in any way. It's only affecting worker objects and
nothing else.
Common device commands now available to test jobs
=================================================
LAVA already supported configuring per-device custom commands that can be used
from test jobs, like this:
```yaml
actions:
# ...
- command:
name: my-custom-command
```
With this release, the following built-in commands can also be used: `pre_power_command`,
`pre_os_command`, `power_on`, `power_off`, and `hard_reset`. Although lab admins can
configure these commands to be anything, the following table describes their usual
semantics:
| Command | Usual meaning |
| :------------------ | :------------------------------------ |
| `pre_power_command` | Turns USB OTG port ON |
| `pre_os_command` | Turns USB OTG port OFF |
| `power_on` | Turns power to the board ON |
| `power_off` | Turns power to the board OFF |
| `hard_reset` | Turns power to the board OFF, then ON |
No extra configuration is required: these commands are already required to be defined
for each device, and are now made available to the "command" action without need to
explicitly add them to the custom commands configuration.