Hi folks,
The 2020.04 tag has been pushed to master on git.lavasoftware.org.
.deb packages have been built in GitLab CI and are published at
https://apt.lavasoftware.org/release
Docker images for amd64 and arm64 have been built in GitLab CI and
are available from
https://hub.lavasoftware.org/
and
https://hub.docker.com/u/lavasoftware
Changes in this release
=================
Upgrading
=========
Device-type templates
---------------------
Device-type templates as been moved from
`/etc/lava-server/dispatcher-config/device-types/` to
`/usr/share/lava-server/device-types/`.
Database migrations
-------------------
A new `version` field as been added to the `Worker` model. The field will
be used to track remove dispatcher version and check for incompatibilities
between master and dispatcher.
Device-types
============
New device-types
----------------
New supported devices:
* Fixed Virtual Platforms
* fsl-ls1043a-rdb
* fsl-lx2160a-rdb
* fsl-s32v234sbc
* imx6q-var-dt6customboard
* imx6q-sabresd
* imx7d-sdb
* imx8mm-ddr4-evk
* imx8mm-evk
* r8a7795-h3ulcb-kf
* sun8i-h3-bananapi-m2-plus
Fixed Virtual Platforms
-----------------------
FVP jobs can now be ran in LAVA. LAVA will execute FVP devices inside Docker
containers that should be provided by job submitter.
A job definition would be:
```yaml
- deploy:
to: fvp
images:
bl1:
url: https://example.com/fvp/bl1.bin
[...]
ramdisk:
url: https://example.com/fvp/ramdisk.img
- boot:
method: fvp
docker:
name: "foundation:11.9"
local: true
image:
/opt/model/Foundation_Platformpkg/models/Linux64_GCC-6.4/Foundation_Platform
version_string: 'ARM V8 Foundation Platformr0p0 [^\\n]+'
timeout:
minutes: 7
console_string: 'terminal_0: Listening for serial connection on port
(?P<PORT>\d+)'
arguments:
- "--cores=4"
- "--no-secure-memory"
- "--visualization"
- "--gicv3"
- "--data={BL1}@0x0"
- "--data={FIP}@0x8000000"
- "--data={IMAGE}@0x80080000"
- "--data={DTB}@0x82000000"
- "--data={RAMDISK}@0x84000000"
- "--block-device={ROOTFS}"
prompts:
- 'root@genericarmv8:~#'
```
To use this new device-type, the job definition should define a
[deploy](https://docs.lavasoftware.org/lava/actions-deploy.html#to-fvp)
and a
[boot](https://docs.lavasoftware.org/lava/actions-boot.html#fvp)
action.
More information in the [FVP documentation](
https://docs.lavasoftware.org/lava/fvp.html).
UBoot USB Mass Storage
----------------------
LAVA dispatcher will now use `bmaptool` instead of `dd` to flash images. The
layout is computed right before the flash.
Docker test action
==================
The docker test action introduced in LAVA
[2020.02](https://git.lavasoftware.org/lava/lava/-/wikis/releases/2020.02/)
has
been improved.
Serial connection to the device
-------------------------------
The connection commands are now exposed in the test environment as
`LAVA_CONNECTION_COMMAND` and `LAVA_CONNECTION_COMMAND_*`.
For example the given connection commands:
```jinja
{% set connection_list = ['uart0', 'uart1'] %}
{% set connection_commands = {'uart0': 'telnet localhost 4002', 'uart1':
'telnet 192.168.1.200 8001'} %}
```
Will be exported as:
```shell
LAVA_CONNECTION_COMMAND='telnet 192.168.1.200 8001'
LAVA_CONNECTION_COMMAND_UART0='telnet localhost 4002'
LAVA_CONNECTION_COMMAND_UART1='telnet 192.168.1.200 8001'
```
Waiting for device on USB made optional
---------------------------------------
By default, the docker test action will wait for the device to be connected
and exposed to the host via its USB OTG port.
In some use cases, however, you need to e.g. interact with u-boot before a
USB OTG port is enabled, so waiting on the device coming up on USB would
not work. To avoid waiting for the USB connection, you can specify the
`docker.wait.device` parameter set to false:
```yaml
actions:
# ....
- test:
docker:
image: my-image
wait:
device: false
# ....
```
Device type templates
=====================
The default device-type templates has been moved from
`/etc/lava-server/dispatcher-config/device-types` to
`/usr/share/lava-server/device-types/`.
When rendering device dictionaries, LAVA will use templates in
`/etc/lava-server/dispatcher-config/device-types` and then fallback to
`/usr/share/lava-server/device-types/`. This mechanism allow admins to
override
device-type templates.
When using the APIs to override device-type templates, LAVA will write the
template into `/etc/lava-server/dispatcher-config/device-types`.
Compression formats
===================
[zstd](https://facebook.github.io/zstd/) as been added to the list of
supported compression formats.
In order to use it in a job definition you should use:
```yaml
actions:
- deploy:
rootfs:
url: https://example.com/rootfs.ext4.zst
compression: zstd
```
The `zstd` package should be installed on the dispatcher. This is already
the case for the lava-dispatcher docker image.
Postprocessing images with docker
=================================
This release adds support for postprocessing downloaded images using
user-provider docker containers. To make use of this feature, you need to
use the new ***downloads*** deploy target. Example:
```yaml
actions:
- deploy:
to: downloads
images:
# ...
postprocess:
docker:
image: my-image
steps:
- /path/to/my/postprocessing/script
```
The provided docker image will be run with the download directory as
current working directory, and the commands listed in `steps` will be
executed. Any changes that the commands make to the downloaded images are
persisted for later actions, and this includes not only modifying the
existing images, but also creating new files in there.
To make use of any files left in the downloads directory, you need a second
deploy action, which can refer to files in the downloads directory using
the ***downloads://*** pseudo URL scheme. Example:
```yaml
- deploy:
to: fastboot
docker:
image: my-adb-fastboot-image
images:
ptable:
url: downloads://ptable-linux-8g.img
reboot: hard-reset
boot:
url: downloads://boot.img
reboot: hard-reset
system:
url: downloads://rpb-console-image-lkft.rootfs.img
apply-overlay: true
```
Interactive non-exiting commands
================================
Interactive test received a new field: ```wait_for_prompt```. It defaults
to True but can be set to False in the job definition. Idea behind this
feature is to allow for non exiting commands to work in non-posix shells.
Example can be invoking fastboot in u-boot shell and flashing the board
from the subsequent test action
```yaml
- test:
timeout:
minutes: 10
interactive:
- name: erase-emmc
prompts: ['=> ']
script:
- command: mmc dev 1 0
name: mmc_dev
successes:
- message: mmc1(part 0) is current device
- command: mmc rescan
name: mmc_rescan
- command: mmc erase 0 0x400000
name: mmc_erase
successes:
- message: "4194304 blocks erased: OK"
- name: fastboot
prompts: ['=> ', '/ # ']
script:
- command: env default -f -a
name: env-default
successes:
- message: Resetting to default environment
- command: setenv partitions $partitions_android
name: setenv-partitions
- command: fastboot 1
success:
- message: "\n"
wait_for_prompt: false
```
Boot QEMU from docker image
===========================
We now allow users to provide their own docker image that LAVA will use to
start QEMU.
This will allow users to use recent QEMU version and to test QEMU itself.
Job definition schema example:
```yaml
- boot:
method: qemu
timeout:
minutes: 2
media: tmpfs
docker:
image: my-qemu-image
binary: /usr/bin/qemu-system-x86_64
prompts:
- "root@debian:"
auto_login:
login_prompt: "login:"
username: root
```
API
===
New certificate endpoints
-------------------------
We've added a support for downloading master certificate and for uploading a
worker certificate to master. This will make setting up a remote worker a
bit
easier for the administrators.
New endpoints available for REST API:
* ```/api/v0.2/workers/${hostname}/certificate/```
* ```/api/v0.2/system/certificate/```
The first one excepts both GET and POST requests so the user can also
download
the certificate for worker for double checking, while the second one can be
used only via GET request and will download the master public key.
New methods available for XMLRPC API:
* #scheduler.workers.get_certificate($hostname)
* #scheduler.workers.set_certificate($hostname, $key)
* #system.get_master_certificate()
They mirror the REST API calls described above.
Authorization
=============
New permissions backend for workers
-----------------------------------
The worker model is now protected by the "new" authorization system. You can
find settings in the admin section for managing individual workers.
Only `change` permission is used for workers, in order to allow
administrators
to control who can upload worker certificate, environment and
configuration files.
Note that the worker authorization settings will **NOT** affect the device
or
device type authorization in any way. It's only affecting worker objects and
nothing else.
Common device commands now available to test jobs
=================================================
LAVA already supported configuring per-device custom commands that can be
used
from test jobs, like this:
```yaml
actions:
# ...
- command:
name: my-custom-command
```
With this release, the following built-in commands can also be used:
`pre_power_command`,
`pre_os_command`, `power_on`, `power_off`, and `hard_reset`. Although lab
admins can
configure these commands to be anything, the following table describes
their usual
semantics:
| Command | Usual meaning |
| :------------------ | :------------------------------------ |
| `pre_power_command` | Turns USB OTG port ON |
| `pre_os_command` | Turns USB OTG port OFF |
| `power_on` | Turns power to the board ON |
| `power_off` | Turns power to the board OFF |
| `hard_reset` | Turns power to the board OFF, then ON |
No extra configuration is required: these commands are already required to
be defined
for each device, and are now made available to the "command" action without
need to
explicitly add them to the custom commands configuration.
The changelog is also available in the wiki:
https://git.lavasoftware.org/lava/lava/-/wikis/releases/2020.04
Thanks
--
Rémi Duraffort
LAVA Architect
Linaro
Hello,
As you know, we are trying to release a new version of LAVA every month.
We are delaying this month release (2020.03) due to some issues and crashes
that we found in the last days.
Issues where found on staging.vlo, lavafed and meta-lava, proving that the
CI is working.
Where are currently working on fixing the issues and proving that LAVA is
ready for release.
The next release should be ready for the end of next week and would be
called 2020.04.
Rgds
--
Rémi Duraffort
LAVA Architect
Linaro
