Hello,
following the release of LAVA 2019.03, we released lavacli v0.9.5
This release brings one new command: "lavacli jobs validate"
This command send the given job definition to LAVA (version >= 2019.03).
LAVA will check the definition against the schema and report any errors.
As usual, lavacli could be installed from:
* pypi (https://pypi.org/project/lavacli/)
* debian (https://tracker.debian.org/pkg/lavacli) (coming soon).
Cheers
--
Rémi Duraffort
LAVA Team, Linaro
The 2019.03 tag has been pushed to master on git.lavasoftware.org.
Packages have been built in GitLab CI and are available at
apt.lavasoftware.org/release. Docker images for amd64 and arm64 have
been built in GitLab CI and are available from hub.lavasoftware.org
and https://hub.docker.com/u/lavasoftware .
Changes to LAVA UI configuration
================================
https://master.lavasoftware.org/static/docs/v2/advanced-installation.html#c…
Initial settings for a LAVA instance change over time as the
requirements change and dependencies improve internal security
implementations. Most instances will need some adjustment to the
apache configuration for the main LAVA UI in
/etc/apache2/sites-available/lava-server.conf and the LAVA packages do
not attempt to update this file automatically once admins have made
changes. Admins should therefore subscribe to the lava-announce
mailing list and make changes using separate configuration management.
apache2
=======
Browser caching can be improved by enabling mod_header in Apache to
allow LAVA to send appropriate cache control headers as well as mod
proxy and mod proxy_http:
$ sudo a2enmod header
$ sudo a2enmod expires
$ sudo service apache2 restart
LAVA server settings changes
============================
Django requires the allowed hosts to be explicitly set in the LAVA
settings, as a list of hostnames or IP addresses which LAVA is
allowed/expected to use. If this is wrongly configured, the UI will
raise a HTTP 500 error and you will get information in the output of:
lava-server manage check --deploy
or in
/var/log/lava-server/django.log
For example, /etc/lava-server/settings.conf for
https://lava.codehelp.co.uk/contains:
"ALLOWED_HOSTS": ["lava.codehelp.co.uk"],
See also
https://docs.djangoproject.com/en/1.11/ref/settings/#allowed-hosts
It is also important to enable ProxyPreserveHost in
/etc/apache2/sites-available/lava-server.conf
ProxyPreserveHost On
In some situations, you may also need to set USE_X_FORWARDED_HOST to
False in /etc/lava-server/settings.conf
See also
https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-USE_X_FORW…https://git.lavasoftware.org/lava/lava/merge_requests/400
python3-junit.xml
=================
python3-junit.xml >= 1.8 is required for REST API calls when running A
suitable package has now been backported to provide this functionality
for Stretch.
https://packages.debian.org/stretch-backports/python3-junit.xml
Note: there is a very similarly named package which will **not** work
(python3-junitxml). Look for the dot in the package name of the
correct package: python3-junit.xml
When installing, remember to allow stretch-backports:
$ sudo apt -t stretch-backports install python3-junit.xml
https://git.lavasoftware.org/lava/lava/merge_requests/357
Bump version of django-tables2
==============================
With 2019.01 and later, django-tables2 version 1.14 does not pick up
the customisation of the tables, so orderable icons and search forms
and pagination are missing.
It was not practical to backport django-tables2 to stretch-backports
due to complexities in the build dependencies of the package. So this
update has been made available via apt.lavasoftware.org
Stretch users will find a new version of python3-django-tables2 will be
installed alongside 2019.03.
https://git.lavasoftware.org/lava/lava/merge_requests/390
Prevent action.timeout exceeding job.timeout
============================================
This change affects the test jobs you submit to LAVA. The test job will
warn if any one action timeout exceeds the overall job timeout. LAVA is
adopting a stricter submission schema and this is the first common issue.
If you have a repository containing a number of test jobs, you can
check all of the jobs in one go by passing the path to a script in the
lava-common package.
$ sudo apt install lava-common
$ /usr/share/lava-common/lava-schema.py jobs ./*.yaml
You may want to edit the timeouts when resubmitting older test jobs.
When running /usr/share/lava-common/lava-schema.py, there may be a
range of other errors reported too. In future releases, LAVA will no
longer silently ignore unknown or obsolete YAML blocks. If there is
an error in the submission, the expanded schema will report it.
It is worthwhile running /usr/share/lava-common/lava-schema.py
against the output of all your test job submission templates. If you
use templating and CI, then you could add the check to the CI of each
change to the templating. Just ensure that the lava-common package is
upgraded to the same version as the instance to which the test jobs
would be submitted.
In a future release, this expanded schema check will be included into
lavacli, lava-server manage and as an XMLRPC call which can be used
before the submission. For now, only the action.timeout exceeding
job.timeout warning is being enabled in 2019.03.
https://git.lavasoftware.org/lava/lava/merge_requests/413
Changes in this release
=======================
60b9c653f checks: fix crash if the cmdline is not in the expected format
86cfe1ac2 Update reprepro-release.sh
6129fe7c0 tweak reprepro-release for new directory
b0cf8d779 rzn1: allow to boot using a fit image available over tftp
3e4faf6e1 Add qcs404-evb-1k-specific health check and fix a typo
9fd571b3e Fix crash in "check --deploy"
07a89fab3 device-type: Add Qualcomm's qcs404-1k and qcs404-4k
dcf3f2c6c debian: lava meta pkg: allow to use chrony instead of ntp
2d8e09126 (codehelp/master) schema: improve schema validator
8fa55d888 xmlrpc: allow to validate job definition schema
5aa407f04 Add default volumes for lava-slave-docker types
8f2a8cbe6 schema: move validate function to lava_common.schema
0b38b5805 Update copyright years across the LAVA source tree
5c17463e1 Package lava-schema.py in lava-common
a846ad088 Provide symlink for favicon
785906e1b Make timeout check non-fatal
6373b6869 lava-schema: allow to read from stdin
25812ecc7 schema: allow for any objects for test def params
861ec2fb3 Revert change to USE_X_FORWARDED_HOST
f26f61ee7 Avoid spurious boot command list warning
4915231e4 Skip analysis jobs on tags
d53814cc0 Add docs on advanced Apache and container usage
b42520584 Mention lava-lxc-mocker in docs
93828f63d device-types: add Intel upsquare
170dc7547 device-types: add pine64+
f1bbb9bb8 lava-server-gunicorn: allow to set the bind address
354077cb4 Schema: handle recovery deploy and boot methods
ae39f9004 Schema: add deploy to "download" method
7b9119bd7 CI check more sample jobs
168c4db20 Bump version of django-tables2 backport
1b0749092 Fix more sample jobs and ad schroot boot
e30c2f03d Fix typo
ea2987a08 Fix error in maintenance breadcrumb
291aa4ddb Make kernel messages wait sensitive to force_prompt
f3adeb798 Add cls.LT and cls.EXACT to models.DateTimeField
84ad3683a Check health-check YAML and schema
9299d5263 REST API: test /junit
1d5ea3cf7 Require python3-junitxml >= 1.8 for REST API calls
dc5a6fe48 templates: handle multinode vs singlenode
5f01a79c6 CI: run pylint to find common errors
00db03351 Fix some crashes found by pylint3
5a92736ad CI: test templates against the schema
3b5c9d5ef Add extra U-Boot bootloader error
2929ba8f3 Raise the codeclimate complexity threshold
d84756688 apache2: send the right cache-control headers when available
3cc42420f rzn1: do not set uboot_needs_interrupt explicitely
656666cb7 maintenance: fix crash after new state machine
00cea9df4 Device/Worker: remove deprecated method
576146730 Fix missing imports
32960bdd5 Handle duplicate listings in requires
19fdfd969 Move get_domain() to dbutils
01d354d1b img2simg and simg2img now available in a new package
bb62625f3 Fixup changes in the bandit overrides
ae25f1fe1 Update copyright for recent changes
4c33f7a54 Wait for the prompt when using shell on DUT
5662f734a code_quality: install radeo in the setup step
19bda4ac5 Support using radon to output code-climate data
489bc8416 schema: check namespace syntax
6406e1368 Improve inline test schema
0ae489101 Schema: check that jobs with secrets are not public
36d7f4ebb Check that timeouts are shorter than job timeout
fd225060e Improve schema validator
3a2fc459b Prevent action.timeout exceeding job.timeout
e6bfc3f43 Allow check --deploy inside Docker
69053fb82 Ignore DeprecatedWarnings on buster in CI
01fa79469 ci-run: use pytest-3 by default
075bf1f28 Fix crash when pre-test install failed
0c1372f9d Fix crash with older python3-tap versions
02097dd41 Fixup 4ebbc234 - put new test in correct location
feff6c914 CI do not run analyze for tags
9d4a91251 Fix documentation
2354f3899 doc: mention the minimal boot method
354f33ac2 imx8m device type restructuration.
4e1090982 base.jinja2: Allow interrupt-newline to be disabled in u-boot
958068fc6 Fix more sample job syntax
27fdc5107 device-types: add hsdk
dd4663b0a multinode: use character_delay
4ebbc234b power: use run_cmd instead of run_command run_cmd: accept
command as a string
9d9650939 Add missing pytest dependency for autopkgtest
d5bf2c248 Fix typo in f6bb9de2
b3eefdedb Fix missing change which breaks Debian CI testing
> Steve, for the LAVA team
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
Changes to LAVA UI configuration
================================
https://master.lavasoftware.org/static/docs/v2/advanced-installation.html#c…
Initial settings for a LAVA instance change over time as the
requirements change and dependencies improve internal security
implementations. Most instances will need some adjustment to the
apache configuration for the main LAVA UI in
/etc/apache2/sites-available/lava-server.conf and the LAVA packages do
not attempt to update this file automatically once admins have made
changes. Admins should therefore subscribe to the lava-announce
mailing list and make changes using separate configuration management.
apache2
=======
Browser caching can be improved by enabling mod_header in Apache to
allow LAVA to send appropriate cache control headers as well as mod
proxy and mod proxy_http:
$ sudo a2enmod header
$ sudo a2enmod expires
$ sudo service apache2 restart
LAVA server settings changes
============================
Django requires the allowed hosts to be explicitly set in the LAVA
settings, as a list of hostnames or IP addresses which LAVA is
allowed/expected to use. If this is wrongly configured, the UI will
raise a HTTP 500 error and you will get information in the output of:
lava-server manage check --deploy
or in
/var/log/lava-server/django.log
For example, /etc/lava-server/settings.conf for
https://lava.codehelp.co.uk/ contains:
"ALLOWED_HOSTS": ["lava.codehelp.co.uk"],
See also
https://docs.djangoproject.com/en/1.11/ref/settings/#allowed-hosts
It is also important to enable ProxyPreserveHost in
/etc/apache2/sites-available/lava-server.conf
ProxyPreserveHost On
In some situations, you may also need to set USE_X_FORWARDED_HOST to
False in /etc/lava-server/settings.conf
See also
https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-USE_X_FORW…https://git.lavasoftware.org/lava/lava/merge_requests/400
python3-junit.xml
=================
python3-junit.xml >= 1.8 is required for REST API calls when running A
suitable package has now been backported to provide this functionality
for Stretch.
https://packages.debian.org/stretch-backports/python3-junit.xml
Note: there is a very similarly named package which will **not** work
(python3-junitxml). Look for the full stop in the package name of the
correct package: python3-junit.xml
When installing, remember to allow stretch-backports:
$ sudo apt -t stretch-backports install python3-junit.xml
https://git.lavasoftware.org/lava/lava/merge_requests/357
Bump version of django-tables2
==============================
With 2019.01 and later, django-tables2 version 1.14 does not pick up
the customisation of the tables, so orderable icons and search forms
and pagination are missing.
It was not practical to backport django-tables2 to stretch-backports
due to complexities in the build dependencies of the package. So this
update has been made available via apt.lavasoftware.org
Stretch users will find a new version of python3-django-tables2 will be
installed alongside 2019.02.
https://git.lavasoftware.org/lava/lava/merge_requests/390
Prevent action.timeout exceeding job.timeout
============================================
This change affects the test jobs you submit to LAVA. The test job will
warn if any one action timeout exceeds the overall job timeout. LAVA is
adopting a stricter submission schema and this is the first common issue.
If you have a repository containing a number of test jobs, you can
check all of the jobs in one go by passing the path to a script in the
lava-common package.
$ sudo apt install lava-common
$ /usr/share/lava-common/lava-schema.py jobs ./*.yaml
You may want to edit the timeouts when resubmitting older test jobs.
When running /usr/share/lava-common/lava-schema.py, there may be a
range of other errors reported too. In future releases, LAVA will no
longer silently ignore unknown or obsolete YAML blocks. If there is
an error in the submission, the expanded schema will report it.
It is worthwhile running /usr/share/lava-common/lava-schema.py
against the output of all your test job submission templates. If you
use templating and CI, then you could add the check to the CI of each
change to the templating. Just ensure that the lava-common package is
upgraded to the same version as the instance to which the test jobs
would be submitted.
In the next release, this expanded schema check will be included into
lavacli, lava-server manage and as an XMLRPC call which can be used
before the submission. For now, only the action.timeout exceeding
job.timeout warning is being enabled in 2019.02.
https://git.lavasoftware.org/lava/lava/merge_requests/413
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Hello,
following the release of LAVA 2019.01, we released lavacli v0.9.4
This release brings two new commands:
* "lavacli utils templates render" to render jinja2 templates
This will help users/admins to debug jinja2 templates issues
* "lavacli jobs queue" to show the current queue for the requested
device-types
This is only available since LAVA 2019.01
As usual, lavacli could be installed from:
* pypi (https://pypi.org/project/lavacli/)
* debian (https://tracker.debian.org/pkg/lavacli)
Cheers
--
Rémi Duraffort
LAVA Team, Linaro
The 2019.01 tag has been pushed to master on git.lavasoftware.org.
Packages have been built in GitLab CI and are available at
apt.lavasoftware.org/release. Docker images for amd64 and arm64 have
been built in GitLab CI and are available from hub.lavasoftware.org
and https://hub.docker.com/u/lavasoftware
New bootloader testing and recovery support
==================================
The recovery mode support already implemented for the hikey 6220 has
been used to provide recovery mode operations for the X15 by Milosz
Wasilewski to support bootloader testing with automated recovery.
Dropped default patterns and fixup dicts
===============================
Omits the V1 compatibility constants for default pattern and fixup
dict. Patterns and fix up dictionaries are now only available in Lava
Test Definition 1.0 test shells if the test writer explicitly adds the
parse: block to the test definition. The documentation has been
updated.
Ensure lxc-templates is installed
=========================
To use lxc, the recommended lxc-templates package must be installed to
provide the template files. When installing lava-dispatcher in Docker,
it is now useful to specify the lava-lxc-mocker package as the updated
LXC packages will fail to install inside Docker.
Final black changes
================
All LAVA Software Python code in lava.git has now been formatted using
black and the GitLab CI ensures that this formatting is retained. If
you are developing code to be merged into lava.git, ensure that you
have black installed and run it as the final step before contributing
your changes upstream or the CI will fail.
REST API using django-restframework
=============================
Added REST API for LAVA, including ability to export results in JUnit and TAP13.
Non-POSIX test action
=================
Add a new test method for non-posix shells. The test writter is able
to launch any command on the DUT and is responsible for giving the
right set of patterns that match failure and success.
Add a new boot method called "bootloader"
=================================
This method allows to boot the DUT, interrupt the bootloader and jump
into the bootloader shell.
Changes in this release
==================
ec7875b05 Enhance the Docker admin documentation
ffe2c0751 Build architecture aware docker images
e0c651aae Update docs for development changes
c0eeb4b28 schema: allow for namespaces with interactive tests
aa94f3ac4 Allow to set uboot_needs_interrupt in the device dict
79fb10a12 Fix debian installer deploy method
8b6b53be0 Allow to set fastboot deploy uboot commands
3975eb4f4 doc: add documentation about x15-bl device-type
b87dc6816 scheduler: add x15-bl device type
5d8227c91 scheduler: add recovery support for base-uboot.jinja2
4ddd44ad6 Tweak the development process doc
2bbf5c6e5 Update git repository links
1a9f51494 Drop bashism in vland helper scripts
7ebb3feab Retrieve the vland version in the protocol
f2f933399 Improve docs on health check setup actions
1e596b6ff Improve server|logs|publisher command line parser
68dd8ff79 device-types: allow to set boot_character_delay in device dict
92fa46213 scheduler: randomise the device list
c0c3eb0ef lava-modules.conf should be in /etc/modprobe.d/
2412366c9 Fix sample jobs timeouts
68edafac9 CI: install minicom for mps2plus tests
f3cbdac58 Add lava-server manage command for site
cc16e4b38 mps2: fix validation and documentation
32d18fa3b device-types: add rzn1 template
7a992201c device-types: qemu: Add support for MIPS/PPC/PPC64
e114b354e qemu: Remove root_partition from sample jobs
407d857f9 Fix sample job syntax
049eb5afc sample jobs: move kernel type to the deploy action
25f37dc62 vemsd: recovery_image is mandatory
596aa3b29 Enforce realpath for which check
68b8e54b1 doc: add information about SQUAD to custom-results-handling
1282c4a25 Remove leftover from py2 to py3 conversion
dd76f0797 Add voluptuous to lava-common dependencies
8041e1012 Rework interactive test
f6bb9de2f Improve job page rendering with large jobs
e9fa46977 Improve schema validator and move to lava_common
f116ef3af Add check-devices.py to lava-dev
02e4d9e88 Fix sample jobs syntax
b4d84f82f Drop default patterns and fixup dicts
9fbcca0e4 Separate device and device-type as distinct types
fb4d5785e Action: name is a class variable
e990310e0 Remove deprecated functions since django 1.9
eb25d433e Reorder middlewares according to django default config
ebf17cbe6 Remove deprecated module
8a317b0b2 Record JobError in validate()
3bc657e7b Fix sample jobs syntax
b7eb162a1 Ensure lxc-templates is installed
a3524ce9a Expand minimal configuration for connections
ef2fa8594 Results table: decrease the number of sql queries
4df8e8464 Handle downloads using "Content-Encoding: gzip"
f37c03159 Move sample job to the right directory
37a2fe7fa Remove "connection" from job definition
d70e2623c Add checks for the connection type
5260ae1b9 doc: fix broken links
33918bcb4 Update ./ci-run for lava_common/tests change
5af8543b1 Simplify docker boot command line build
f6be3692c Update emphasis lines of literalinclude blocks
da1d472b5 Fix problems with the purge maintainer script
28afc08eb Remove add-header mentions
832fb942b Use render() shortcut
7ef4ac466 Fix bandit warnings after black formatting
d483d94a5 Make flasher deploy retriable
7f3e23ee1 Do not add yaml_line to the job definition
c76620a55 sample jobs: remove mkimage_arch
5be4be588 Move test_info to the job level
9eb721166 'nfs_url' option does not exist anymore
7c347e04e Fix the increased SAST count from formatting changes
017285ce5 Remove outdated README file
b471ee62f Unify test directory name
46f298251 fix prospector warnings
f7973476b Final black changes for lava_dispatcher
6cae9e765 CI: fix black check
816a47b9b Fix typo in check-devices error message
bc844be5c Apply black to lava_dispatcher
40e6a25f8 Apply black to lava/
93a70fcfc Update restore instructions
4ef3b776a Test repeat fix
01432ba06 Convert test_repeat to pytest
f9e782cc8 Fix repeat logic
72d563661 Update advice on restoring backups
bee7cc6da Update documentation example to use correct syntax
71276203a Fix inline example test job indent error
3d8cd9e88 Advise on upgrading python dependencies
e22dbf0c3 Fix states in maintenance.py
2cf6b51ab interactive: allow to only wait, not sending any command
bfb7cb47c Fix test stages computation when mixing test types
8bdc42ef3 Make StdoutTestCase.pipeline_reference a class method
989b21688 Search the queue by requested_device_type
32a4c821b notification: protect against potential crashes
b7254aec6 XMLRPC-API: decrease the number of sql requests
e65cfdc09 device-types: qemu: Add support for SPARC/SPARC64/MIPS64
ade4b7e0c Apply black to lava_scheduler_app
95aa92f9f Apply black to doc/
54c18a449 Apply black to share/, setup.py and version.py
c74c15d06 Simplify black listing
fd04401a9 Apply black to lava_results_app
8f100d82b Apply black to lava_server
7aa96e077 Apply black to lava_common
5e6e902fa Update documentation for new test action support
bfa253f9b Document the docker deploy
84d202b28 Device table: tag with <i> when worker is undefined.
009e21b53 doc: explain how to serve LAVA under /lava
d86d457f7 device-types: add meson-gxl-s805x-p241
8fb830b3d REST API: allow to export results in JUnit and TAP13
b50800bfc Add a new test method for non-posix shells
ce2627669 Let udev_trigger use the network from udev
172c6f22c Add a new boot method called "bootloader"
4e7248342 Add a qemu-aarch64 device-type
c715f5252 lava_results_app: fix exception when no action_metadata
229242900 Fix deprecation warnings
6cc1c7966 Set ProxyPreserveHost on.
b801a30f4 Create an out-of-tree symlink for djangorestframework
e6678963c Fix rest_framework static symlink
cac2cbb7f Allow dependencies of reprepro-master on schedules
438e23cab Enable reprepro-master
9788877eb checks: log an error when settings.conf is invalid
ad6a2c61d CI: only run deploy jobs for scheduled jobs
f4530df6b TC2 device jinja changes
349ee8175 Remove unused command migrate-lava
6690af7cf CI: skip some tests when lxc-start is not installed
cd7d9e21b Fix crash if dpkg-query does not know about a file
b9cb118ef Add docs on vemsd and mps deployments
d0fd22894 Update the docker-admin page for new links
9bac98720 REST API reduce the number of SQL requests
4bfad2969 Copy lava-lxc-mocker in the publish stage
dacd2a41c Fix lava_rest_app support in ci-run
421e5283d CI: use new sast syntax
0d75252a4 Apply black to lava_server files
b28764d28 Fix unit test exception handling
8e38d322e Add REST API for LAVA
5a55d6646 Update the release repository only on a tag
b55b1c1ba Report the version of tools
3e8f0eb5e Replace with a more pythonic way.
cef780613 Fix doc typo causing build error.
62150f062 Apply black to all migrations
7d040ca09 Optimise to reduce the compare cost.
688df87de Fix: MPS Serial Buffer
8a3d6a587 monitor: fix accepts logic
ade6320a2 Replace the expired job link with a valid one.
bfa558f30 CI build aarch64 docker image using debian 10
ef1c54427 Make navbar and userlinks seperate templates
9270725c9 Correct the path of file "commands.py"
4e7d05696 Tidy up ci-run
73d3a74a4 Ensure sample_jobs validate against the schema
0be0739ab Extend timeouts of example test jobs
b024c2c69 Skip OverlayAction if the overlay is not needed
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Hi,
In founding the LAVA Software Community Project, the team planned to
open up LAVA development more. As already announced by Neil in
September, we have already moved our infrastructure to a GitLab
instance and LAVA developers and users can collaborate there. [1]
The next step in our process is to also open our regular development
design meetings to interested developers. The LAVA design meeting is
where the team gets together to work out deep technical issues, and to
agree on future development goals and ideas. We run these as a weekly
video conference using Google Hangouts Meet [2], We now wish to
welcome other interested developers to join us there too, to help us
develop LAVA.
Summaries of the meetings will be posted regularly to the lava-devel
mailing list [3], and we encourage interested people to subscribe and
discuss LAVA development there.
[1] https://git.lavasoftware.org/
[2] https://meet.google.com/qre-rgen-zwc
[3] https://lists.lavasoftware.org/mailman/listinfo/lava-devel
Cheers,
--
Steve McIntyre steve.mcintyre(a)linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
We've done a little bit of advance planning and there are now three
milestones which developers will be using to schedule upcoming
releases.
https://git.lavasoftware.org/groups/lava/-/milestones
Issues and merge requests have been assigned to the relevant milestone.
All dates are approximate.
We will also be announcing (here) when the next release would include
database migrations so that admins have more notice of scheduled
downtime. When a milestone includes database migrations, other changes
which also include a database migration will be moved to that
milestone so that admins can minimise total downtime.
The next release is expected to include the changes from the 2018.12 milestone.
The 2019.02 milestone already includes database migrations.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Changes in the master branch (and daily build packages) now check the
ALLOWED_HOSTS setting for each master in your instance(s) in
/etc/lava-server/settings.conf
Please check your /etc/lava-server/settings.conf support for
ALLOWED_HOSTS in Django as this will be required to upgrade to the
next release of LAVA Software.
Remember to always restart lava-server-gunicorn after any change to
/etc/lava-server/settings.conf
See also https://master.lavasoftware.org/static/docs/v2/pipeline-debug.html#check-la…
and https://master.lavasoftware.org/static/docs/v2/pipeline-debug.html#displayi…
(new help sections on using and checking LAVA setttings.
Django docs: https://docs.djangoproject.com/en/1.11/ref/settings/#allowed-hosts
"""
This is a security measure to prevent HTTP Host header attacks, which
are possible even under many seemingly-safe web server configurations.
...
Django also allows the fully qualified domain name (FQDN) of any
entries. Some browsers include a trailing dot in the Host header which
Django strips when performing host validation.
...
When DEBUG is True and ALLOWED_HOSTS is empty, the host is validated
against ['localhost', '127.0.0.1', '[::1]'].
"""
If this setting is wrong, updated LAVA packages will fail to serve
HTTP and the /var/log/lava-server/django.log file will record messages
like:
ERROR 2018-11-26 18:12:52,091 exception Invalid HTTP_HOST header:
'lava.codehelp.co.uk'. You may need to add
'lava.codehelp.co.uk' to ALLOWED_HOSTS.
(In this case, that was exactly the correct action)
"ALLOWED_HOSTS": ["lava.codehelp.co.uk"], to
/etc/lava-server/settings.conf and restarting lava-server-gunicorn
fixed lava.codehelp.co.uk
See also https://git.lavasoftware.org/lava/lava/issues/173
When checking this change, it is useful to also set DEBUG to true as
Django will then show you all of the headers and environment
variables. Do remember to turn DEBUG back to false because these
variables are a security risk in themselves.
Another setting which might be useful, depending on your configuration:
"USE_X_FORWARDED_HOST": true,
It is safe to make this change in advance of the upgrade of LAVA but
do test on an instance which is as close as possible to the network
configuration of your production instance(s).
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Main changes in 2018.11
===================
0eb0d3ed7 Set requirements for requests to avoid CVE - as announced
previously python3-requests needs to be updated to protect against a
CVE in that package.
If installing 2018.11 on a Stretch system, ensure you have updated
python3-requests to the version from stretch-backports.
f7bbf6a8 Visibility of worker status effects on device views-
(improved in subsequent commits in this release). Indicate the status
of the worker when displaying information about the device so that it
is clear why the device is still Idle if there is a problem with the
worker.
ac0097c4 Fix 500 when rendering device dict page - if the device
dictionary is invalid.
0f7a7ec8 Prevent index out of range error and 97132e91 Extend 0f7a7ec8
to other command_output comparisons and 06203dfe Add a run_cmd helper
to Action class - the Action run_command support had become complex
and problematic. A new parsed_command has been added and a simpler
run_cmd. In time, existing use of run_command will be phased out in
preference for one of the two new commands. This should make it easier
to spot why certain dispatcher commands failed. This was triggered by
intermittent problems running simg2img and img2simg for AOSP.
4736b01d How to test and recover bootloaders in LAVA -
https://master.lavasoftware.org/static/docs/v2/bootloaders.html
ff37c802 Add docs on criteria for LAVA on other distributions -
https://master.lavasoftware.org/static/docs/v2/debian.html
Issues closed in 2018.11
===================
https://git.lavasoftware.org/lava/lava/issues?scope=all&utf8=%E2%9C%93&stat…
Full changelog
===========
ff37c802c Add docs on criteria for LAVA on other distributions
4736b01dc How to test and recover bootloaders in LAVA
2cae2b413 Fix lava-master crash when device yaml is invalid
570fb324d Allow sdist to be passed to setup.py
51b3fe6f6 Fix missing part of aarch64 stretch deployment
f9a3671c1 docker: check method
563fac8da Enable aarch64/pkg-debian-9 in the build tasks
7e71a56ea flasher: fix substitution when cmd contain whitespaces
5bb71d4a1 Package the requires.py script
d77c8d28f Add CI package script for stretch on aarch64
24eface95 docker: use the new Action.run_cmd helper
ddd986e8e Extend the base poweroff timeout.
9d49da1d4 Remove build from .gitignore
24022feb4 Fix missing return value in run step
b5c51dfa8 Force Juno to use NFS vers=3 and extend power off timeout
7ddae776c run_cmd: make spaces explicit
87f1e01e1 Capture the simg2img and img2simg output and log it
f40f0f2c5 Ensure apt is updated before trying to install
06203dfe2 Add a run_cmd helper to Action class
97132e916 Extend 0f7a7ec8 to other command_output comparisons
f0ebf8025 Do not send "\n" twice
ac0097c42 Fix 500 when rendering device dict page
57d80c2e2 Add deployment builds to snapshot directories.
9a1a6bfb1 Apply black to more files in lava_scheduler_app
c6bfbd5f8 Apply black to lava_dispatcher unit tests
0b615023d version: allow user to specify the branch to use
f7bbf6a8b Visibility of worker status effects on device views
f2d9b658e Apply black to lava_scheduler_app/api and tests.
7980d84c3 docker tag: move branch name to suffix, not prefix
0f7a7ec8f Prevent index out of range error
984164f68 Fix omission in package deployment script
d4b942b71 CI use pytest-3 instead of py.test-3
d77c7080c Fix regression in fb4a88388
504115553 Update docs on local dev builds
fb4a88388 Standardise on dots in the version string
4a1910049 Use the rollback support in requires.py
cb86cc09b Improve error message after 4c71c2ebd1
15d7a905f Include package artifacts into repositories
4c71c2ebd Provide more information in bootloader errors
abff27c8b CI: skip deploy when lavafed is running latest version
f691bc059 CI fix typo for "only.refs"
024b27694 Fix errors in docker-admin page
a937a29e4 CI: add missing requirement to pytest
08ea617a4 vland: fix crash when finalizing
987280afd Fix errors in docker-admin page
89f96b1d9 CI fix deployment script
c5de132b1 Deploy lavafed-master when scheduled
08eab0bcb Fix issues with unit test calculations
7e8a6f444 CI use the new arch specific images
7d2956e43 Sort the --names output of requires.py
0eb0d3ed7 Set requirements for requests to avoid CVE
8cd80dcb2 Build packages and docker images for AArch64
1ab6aaa9c Allow unit tests to run without /sys/class/misc/kvm
83649d77b Port 0e598e63 to the xmlrcp api
6335f73d8 Prevent crash if environment requested for non-POSIX
14b347c51 lava_results_app: convert Decimal objects to string
storing as YAML
26bf0af63 lava_results_app: add test case for YAML Decimal
object conversion
1744ea74c Change ownership of health checks and device-types
32a2a6051 Ignore gitlab-ci files when creating a release tarball
bd224fcd1 Update publish for changes in pkg/docker
2e5e60c9f Drop obsolete gitreview file
2018.11-1 has been uploaded to Debian unstable. This version should
migrate into buster, at which point an upload to stretch-backports
will be made too.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
To avoid CVE-2018-18074, LAVA has bumped the dependency on
python3-requests to 2.20.0
https://git.lavasoftware.org/lava/lava/commit/0eb0d3ed7fa122b08c7ada7d27cfa…
If you are running LAVA on Stretch, you should take advantage of the
security fix by installing python3-requests from stretch-backports.
If you are tracking the daily packages for a staging instance, those
packages now depend on 2.20.0. Installing those packages on Stretch
will require installing python3-requests from stretch-backports. This
may cause automatic upgrades to be unable to upgrade the LAVA
packages.
$ sudo apt -t stretch-backports install python3-requests
If you are running on buster, the new version is already available in
buster and will be upgraded automatically.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
