Hi folks,

The 2024.09 tag has been pushed to master on gitlab.com/lava/lava.
.deb packages have been built in GitLab CI and are published at

  https://apt.lavasoftware.org/release

Docker images for amd64 and arm64 have been built in GitLab CI and
are available from

  https://registry.gitlab.com/

and

  https://hub.docker.com/u/lavasoftware

Changes in this release
==================

# Upgrading

## Database migrations

This release includes migrations for both lava_scheduler_app and lava_results_app. The migrations add unique constraints to fix potential (and rare) race conditions when receiving and parsing job logs.

In order to apply the migrations, the database should not contains TestJobs with duplicated sub_id, TestSets, TestSuites or TestCases. Otherwise the migrations will fail.

An helper script as been added as part of this release to **remove test jobs with such duplicates**. The helper **should** be ran **before** running the migrations.

```shell
lava-server manage check_uniqueness --yes
```

When using `--yes`, the helper script will automatically **remove** the faulty jobs.

# Device-types

## New device-types

* imx8mp-debix-model-a
* imx91-9x9-qsb
* imx93-11x11-frdm
* imx95-15x15-ab2
* imx95-15x15-evk
* imx95-19x19-verdin
* k3-am62p5-sk
* qrb4210-rb2-uboot
* r8a779f4-s4sk
* r9a07g044l2-smarc

# Deploy actions

# Downloads

Make overlay tarball available in downloads. When running postprocess action in "downloads" deployment it is sometimes necessary to use LAVA overlay. Example use case is when running tests on the board without networking. This release makes it possible to, for example, append LAVA overlay to the ramdisk during postprocessing step.

```yaml
actions:
- deploy:
    images:
      image:
        url: 'https://example.com/Image'
      dtb:
        url: 'https://example.com/device.dtb'
      ramdisk:
        url: 'https://example.com/ramdisk.gz'
    timeout:
      minutes: 5
    to: downloads
    postprocess:
      docker:
        image: mkbootimage:master
        steps:
        - mkdir lava_overlay
        - tar xvf overlay*.tar.gz -C lava_overlay
        - "(cd lava_overlay ; find * | cpio -o -H newc -R +0:+0 | gzip -9 >> ../ramdisk.gz)"
        - mkbootimg --header_version 2 --kernel Image --dtb device.dtb --cmdline "earlycon clk_ignore_unused pd_ignore_unused audit=0" --ramdisk ramdisk.gz --output boot.img
```

## Fasboot

Allow fastboot devices to use minimal and u-boot boot methods. Some boards like the Qualcomm robotics platforms can run U-Boot by chainloading with fastboot. Add support for using the U-Boot boot method as a second step after "fastboot boot".

## u-boot

### Device tree overlay

Add support for device tree overlay when deploying to u-boot with tftp. The resource should be called `dtbo`.

```yaml
actions:
- deploy:
    to: tftp
    dtbo:
      url: https://example.com/dtb-overlay

```

The support should be added to the u-boot device-type with:

```jinja
{% set enable_dtbo_support = 'true' %}
```

Also specify the right addresses with:
```jinja
{% set bootm_dtbo_addr = '0x93c00000' %}
{% set bootz_dtbo_addr = bootm_dtbo_addr %}
{% set dtb_base_resize = dtb_base_resize | default(1048576) %}
```

### Skip power reset

Allow to skip resetting the device for u-boot. Allow user to define a fastboot boot action to load u-boot boot image, and then define a u-boot boot action to run u-boot level boot steps.

```yaml
actions:
- boot:
    method: fastboot
    [...]
- boot:
    method: u-boot
    reset: false
    [...]
```

## USB mass-storage

Add a new method to deploy to USB mass storage gadget. This feature requires hardware support on the worker and is currently only available for u-boot DUTs (but can be extended to other DUTs).

The device dictionary should include:

```jinja2
{% set usbg_ms_commands = {
    "disable": ["gadget-ms", "off"],
    "enable": ["gadget-ms", "on", "{IMAGE}"],
} %}
```

The job definition should look like:
```yaml
actions:
- deploy:
    to: usbg-ms
    image:
      url: https://example.com/.../20231109_raspi_4_bookworm.img.xz
      compression: zstd
      format: ext4
      partition: 1
      overlays:
        lava: true
```

# LAVA server

## Rest API

Add a new endpoint to cancel jobs in batch by POSTing to the endpoint the list of job ids.

## Scheduler

When finishing a job, the scheduler will now schedule the next jobs way faster. The process for reporting the end of a job has been improved, reducing the time between two consecutive jobs.


Rgds

-- 
Rémi Duraffort
Principal Tech Lead
LAVA Tech Lead
Automation Software Team
Linaro