For 2017.5, we are planning on changing all of the action names
(self.name) to use hyphens to separate words instead of the current
mix of hyphens and underscores. This is to make future support more
consistent.
All V2 templates currently in upstream will pick up these changes as
part of the review process.
Any V2 templates which are not upstream and which reference timeouts
for specific actions may need adaptation, however, note that 2017.5
also moves a lot of the timeout settings to base.jinja2 anyway.
If you have test jobs which reference specific actions, these will
need changes too.
We're implementing a change in the dispatcher codebase which verifies
that underscore is not used in the self.name of any class inheriting
from Action, so all code reviews will also need updating.
https://review.linaro.org/#/c/19027/
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
This interim release includes a small set of fixes aimed at the LKFT
instance and HiKey support using LXC.
If your instance uses the pre_os_command support for fastboot with LXC
(not just pre_power_command), then note that this support now needs to
be called from within the LXC protocol.
See https://lkft.validation.linaro.org/scheduler/job/885/definition#defline77
for an example.
Changes:
lava-dispatcher (2017.4.post1-1) unstable; urgency=medium
.
* Production hotfix release for LKFT support.
3f99cc3 Add missing super call to WaitUSBDeviceAction
5436658 Fix indent typo
4847bf1 Put device rebooting info message at the correct place.
6b85382 Fix call to lava-lxc protocol for pre-os-command
bdb4d12 Fix test_lxc_api unit test
e9df320 LAVA-920 - Workaround ptable issue in firmware
7b4096c Allow LXC protocol to call pre-os-command
b33d04b Fix OE image support for HiKey.
d714a9b Add missing test shell helpers
9518e74 Add missing import
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
V2 healthcheck changes
======================
https://lists.linaro.org/pipermail/lava-announce/2017-March/000027.html
This support has now been delivered in 2017.4 using the lava-server
manage migrate-health-checks command. If used, the --clean option will
remove the old definition from the database for the device-type.
https://staging.validation.linaro.org/static/docs/v2/healthchecks.html#inde…
There is also a new migration status page to help admins track which
devices still need to be migrated to V2.
Branding message
================
The branding support now includes a message which can be added to the
home page for the instance. For example, a short message to describe
the main purpose of that instance.
Reminder about the LAVA repositories
====================================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
Changes
======
A longer list of changes this time as 2017.3 was skipped to allow for
the Linaro Connect BUD17 conference.
lava-server
===========
7e36443dd Restore UEFI boot method for HiKeys.
6679d548e Fix dragoboard410c boot sequence.
5082adc80 Move anchoring fix for navbar to base template.
187c8520a Allow to override U-boot tftp command
0d28de652 Handle OSError as well as IOError in metadata store
be04868ff Update migration status page for disabled healthchecks
5f689e930 Improve job page
ed4dee272 Expand the docs on arbitrary device_info elements
35a7378e0 Remove references to deprecated health_check_job
5e5e6890c Add a link to the migration page
84f88a4fc Allow to override U-boot usb command
043bfeea4 LAVA-916 - Restore adb connectivity with hikey in V2
4211de051 Don't force {BOOTX} macro in U-boot template
571d71313 Don't force dhcp in U-boot template
b0184585c Add a section on which file is what in /etc/
9701ad6a0 Adding timeouts to Juno template
b2b8d8851 Add content on developer workflow
e36889a81 Revert change for debug filter button
c06ed4a22 Fix bug 2927 - typo in OPTIONS for $MASTER_CERT
cfa0c02c5 Fix navbar problem with in-page anchors.
bd554a2c4 Add docs on boot commands.
624c79ee1 Fix bug #2925 - LDAP Configuration demo mismatch distro.py
1ddde696b Another parent timeout needed for qemu
c3f356c0b Allow boot_qemu_image action timeout to be overriden
b24fe09af Use standard paths to keep the unit tests running
7433bb4e4 Add note on installing developer builds
a0e7756c7 Add a migration status page
ce59699ad Tweak the device type docs
adf670e7d Update copyright year for the docs to 2017
b2428ad8c Small updates to the docs for test development
6610ce36f Allow to import token from another instance
d57e5217b Cleanups for advanced installation docs
0119fc57f Fix job submission page
0c4e68769 Dedupe the index page and add more entries
ec0aa40ad Improve device page
912c174b1 add notes on the new lava-tool features
a662f0ed6 Add a standard NFS test job for panda
e80ed1e13 Update docs on transfer_overlay
b0ae30b56 Update migration for job_status_trigger for backwards compatibility.
f28af7fbe Fix mustang doc examples and add uefi-menu
a755b4b12 LAVA-768 allow to disable hc for a device type
7a05b7941 Add support for a branding message
b18e8d293 log: add a link to the top and to the doc
0fd8c985e fix typo in rst header
75d411f14 Do not send a spurious POST submit request
d7f8d6322 Update docs for change in behaviour of log file UI
acd068173 LAVA-112: display status and role for sub jobs
bf144047f Check V2 devices have valid config before reservation.
0a2da3f92 Skip retired and obsolete device types
c106b17c0 LAVA-913 - Documentation for lxc persistence
14c733d30 Add another index entry for the example first job definition
b8629e0e3 Expand simple admin to include administration outline
43bb9da87 V2 jinja2 template for armada-8040-db
ca05a93d0 Add a command to migrate health checks to the fs
ad55286d6 doc: fix label
0b17a0496 Add qemu NFS docs
a8bf524f4 Update of doc examples
7bc903e70 LAVA-736: move health checks to the file system
1de7ea5b3 LAVA-912 - Document verbose for lxc protocol
643c23efc Remove deprecated (and unused) functions
91cff154b Add notes on when to add more workers
69df43a50 LAVA-904: Improve job page layout
3e3075f2f Reorganise the scheduler app test files
639600952 LAVA-910 allow iteration if vland tag check fails
1ebe9d217 Improve readability of testcase results
b7792c56e fix typo in index syntax
143b5f742 Allow boot_interface override
b85bc07a6 Update default interrupt_* in hi6220-hikey device type jinja2 template
a561326bc Add a test case for secondary connections
d6a441c16 Add Acer Chromebook 13 CB5-311
7ded576cb Add Rada Rock 2 Square
65e2b1e22 Add Renesas R-Car Starter Kit Pro
61edf0180 Add Samsung Chromebook 2
068005a9a Add i.mx6q sabre lite board
a84a509e4 Allow u-boot-interrupt timeouts to be overridden
b7a23dbfd Fix test job timeouts and fix doc
c94238613 Ensure extra_kernel_args is used from job context
77693f080 Clean up error handling in is_deprecated_json method.
8b5fa8130 Add a 'full docs' link to the front page
9ab49a8e9 Allow unit tests to run without kvm module
e47c5c005 Change job_status_trigger field type to a non-deprecated one.
6ec411a55 Tweak the authentication configuration docs
7ee493bb1 Tweak the authentication configuration docs
baa9f5ebb Update references for a permanent location.
da79c4be7 Add available architectures for qemu 'arch' validation.
65e4c35ec Prevent intermittent unit test failure.
7f9b8760d LAVA-215 QEMU NFS support (server side)
3f2f414e2 LAVA-619 document changes for bootz to zimage
8bbbd425f Ensure missing font is available for packaging.
0f2d8a4ff X-axis attribute in charts now use NamedTestAttribute table.
93cc20aa6 Fix export custom queries feature.
a8d6cab43 Display results sorted naturally by key in job logs.
e7aa614c5 Update the default length of the job result table.
e2c07c1ac Fix doc example YAML syntax to read from file
ce59d6b94 Fix sphinx build errors in manpage
63d5f6fa5 Add a check on all templates in source tree
d9eb3e1f3 Ensure character delays are supported by overdrive
75efcb00b Allow repeating test cases
93d620a53 Explain how connection-namespace parameter works.
9041b0dc0 Begin a section on common YAML syntax errors.
6521cc0f8 LAVA-894 - Document lava lxc device add / wait command
ee3e5964f Fix documentation - punctuations, links, bullets.
988d89a34 Add server side support for mustang EFI Grub.
525d9ed47 Fix code blocks in Boot Action documentation.
106f7f4fc LAVA-890 - Allow DUT to stay in Android OS
887175d82 Remove unused images
9f9e8d304 Fix unknown test case handling in tables
bff6ea8ff LAVA-888 - Download button in Device Dictionary page
3314966de LAVA-887 - Introduce fastboot sequence in device type templates
1a1be1ff2 LAVA-867 - Allow fastboot options in device dictionary
d876d1463 Remove legacy nexus deploy document from V2 docs.
2750804af Updating Juno jinja2 template to accommodate vemsd deploy action
2b6d3d277 Deprecate "pipeline-worker" in favor of "workers"
587bc8931 Add a "details" command to "device-type" and "workers"
96761e533 Add a command to manage devices
3b2f27ee0 Encode maintenance state machine in unit tests
730b57b81 Do not put into maintenance retired devices
c0adfd2c2 Add a command to go into maintenance
49abd8bf7 Fix state transition when OFFLINING a device
f337d7d1c Go from OFFLINING to RUNNING when a job exists
39b50ec15 Extend docs on simplistic testing
8e56315eb Workers: allow to set properties
944caa4fd pylint improvements
c42629220 Use continue in a loop, not return
61b6886f9 add notes on debugging multinode as admin
2ed1f35c7 Device tables: add more fields to select_related
d95ad2d5b job logs: Highlight all failing results
ace40b01a Generate less database queries
7fcbd7c68 Allow to use last version of Django Debug Toolbar
61069528d LAVA-862 - Integrate Google Pixel into LAVA V2
e7925e7a1 Update docs on publishing and attaching to tests
da99f08f0 Fix a typo in nexus5x device template used for unit test.
ba5a106db More stdout changes to clean up the test output
c84741460 Skip panda_lxc template check if lxc not installed
6a0a73d36 Fix a typo in documentation.
79ea7cedd Fix looping bug in template
466f1506a LAVA-841 - Document device_info attribute
d420b9d5f LAVA-856 - Integrate nexus5x into LAVA V2
de33008ec Autologin doc update.
lava-dispatcher
===============
7aac7ae5 Fix problem with self.host on secondary connections
530ca366 Ban / in test case names, replace with underscore.
31ee0012 If allow_fail is not set, ensure the command fails
e53a280f Remove old device_type config files which are unused.
80fb3997 LAVA-916 - Restore adb connectivity with hikey in V2
c57a102b Add debug output to the LXC device operations.
7deea0fb Add missing calls to super
341ac593 Writing connection info to namespace data
3ee2af87 Upgrade the boot commands output to info
a1a87daa Fix bug in checking return value of action run_command output.
604323b0 Add a unit test for primary ssh connections
561f78a6 LAVA-914 - Action run_command should return log on allow_fail
a8fb0d9d device-types: add Armada 8040 DB board
98fc1d4d LAVA-911 transfer overlay to the rootfs
e85af0a2 Report versions of critical software on the worker
bdc97bd5 LAVA-901 - Allow lxc debug output if requested in job
93aab7ab Add systemd packages as default for debian and ubuntu templates.
d01f01a3 Allow params for install git repos support
eb821dc8 testdefs: rename the extra results keys
f262b21c testdefs: fix TOCTOU issues with test definitions
07affb55 Print warnings in validate and not in populate
3de0aac9 Make empty revision message more friendly
685eea49 Do not populate append when using tmpfs GuestFS
22a7ba93 Add missing imports
a1baf85e Add missing JobError import.
9e8c3c2f Improve validate and job summary
b1100080 Catch error in secondary connections
7428a9ae Improve git error logs
949b3fcd Rework the timeout settings in job and device conf
6933ac63 download: improve error message
304c39f7 LAVA-852 - LXC fastboot support for persistent devices in V2
fc28aafe Add an error type to the job result
522946b8 Make '/sys/class/misc/kvm' a constant.
f9a1b381 tests: use the DummyLogger whenever needed
ee11c796 tests: use permanent urls
538ff35f Context architecture validation for qemu.
bd88adab Check for enabled kvm module on workers.
28c66090 Fix bug #2898 - Job doesn't exit on fastboot-deploy failure
2879c019 Handle qcow2 conversion errors
7daf65d1 Action: use LAVAError base exception class
603fd049 shell: remove unused exception handling
da97077d Add missing import and remove uneeded one
beccc4da LAVA-215 QEMU NFS support (dispatcher)
c2e3ee34 Use DummyLogger for grub and add .target()
ed3519d2 Get uboot header length from utils.constants
7df2904d If no kernel type given in deploy, use the boot type until deprecated
82237860 Ensure the deployment data shell is exported.
108ef4a3 Add the commit-id to the lava testdef results
1e3dd3ce run: improve error handling
41b08257 lava-dispatch: use the new LAVAError base class
171e8b65 device-types: add Allwinner A23 EVB board
5898747b device-types: add OrangePi PC board
4ffb4c1e Simplify lava-test-shell by removing stdout.log
a5222296 validate: improve error handling
c2df0b20 Remove the message from cleanup()
78c334c2 Simplify test/shell.py check_patterns signals.
68f1c822 test-runner: remove redundant functions and files
3fe2105e Fix bug #2888 - adb root command from job breaks connection to device
22157cd2 Add a base class for LAVA exceptions
44af91c3 Support using UEFI Menu to load Grub using PXE
b667f2fd Use LAVA exceptions
1690ed95 Pylint updates for deploy actions
e538b5d8 LAVA-890 - Allow DUT to stay in Android OS
bbcf76fd Raise a TestError when the test installer fails
0d45d8fa test-runner: use the current shell
27593024 Blacklist brltty on workers
e2593966 Allow reusing of serial connection from another namespace.
cfe9821b LAVA-867 - Allow fastboot options in device dictionary
d5094f28 Fix unknown usage command in lava-test-reference
381eca8d LAVA-881 - Sequence list for fastboot pipeline
e84c00b1 action: be consitent when printing result duration
82d7f82f Print the result of a test run only when avaiable
1d09ece0 Catch python3 errors to clean up output
833d7207 python3: only show requests logs above warnings
5fb1e8a0 rpcinfo: also capture the output
496c7181 Sanitise default mkimage_arch and kernel handling
a6838ecb LAVA-862 - Integrate Google Pixel into LAVA V2
d2f968e3 Follow up to stabilize dragonboard 410c.
8ab18e05 rpcinfo check needs to specifically check NFS service
61777bb0 Refactor wait usb device action to stabilize dragonboard 410c.
e7649e64 Remove references to device_path in device configuration.
3fd11d81 LAVA-856 - Integrate nexus5x into LAVA V2
d970298d Adding new action for monitoring USB mass storage devices via udev
94876fd7 pyocd: allow multiple images to be flashed
26528b0c Detect bootloader resets.
09650ab1 Adding a Versatile Express Firmware deployment action
605fe529 Detect errors during auto login.
00306b79 Job: log the job result as a logger.results
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
2017.4 will deliver support for migrating V2 health checks out of the
database and into /etc/lava-server/dispatcher-config/health-checks/ on
the master. This has several benefits for admins.
* Health checks from 2017.4 onwards will be based on the device
configuration, not the device type. This applies to V1 devices and V2
devices.
* Multiple instances can all have the same V2 health checks controlled
in VCS using configuration management tools.
* V2 devices of the same device-type can have different V2 health
checks according to the Jinja2 template specified in the extends field
of the device dictionary. Submissions to such devices would need to be
managed through device tags.
* Devices which are V1 only (is_pipeline is False in the django admin
interface) will continue to use V1 health checks from the database,
based on the device-type.
* Devices which support V2 (whether these devices are exclusive to V2
or not) will only use V2 health checks from the filesystem, based on
the device dictionary of each device.
This does mean that devices which are currently both V1 and V2 will
need to have a V2 health check from 2017.4 onwards or the device will
no longer run a health check at all. Admins may choose to submit and
monitor regular V1 testjobs until all test jobs for those devices are
V2 but those test jobs will not be able to take the device offline
automatically.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
There is a bug in python-lockfile available from jessie-backports:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858835
This affects fresh installs of LAVA when LAVA is installed directly
from jessie-backports without first being installed in jessie.
The underlying problem is with apt which pulls in the updated version
of packages which have a backport available even if the requested
package does not explicitly ask for the backport of that package.
The problem shows up with lava-coordinator first - the package fails
to install cleanly because the daemon cannot be started correctly.
To fix this problem, simply instruct apt to install python-lockfile
from jessie, not from jessie-backports:
apt install python-lockfile=1:0.8-2
lava-coordinator will then install correctly.
This change will persist over later upgrades as apt does not consider
jessie-backports as upgrades of jessie - except in the case where the
package is not yet installed.
Other daemons also now need to be restarted:
$ sudo service lava-server restart
$ sudo service lava-master restart
$ sudo service lava-slave restart
Future builds for the staging-repo and production-repo will force only
version 1:0.8-2 of python-lockfile as these builds are done on and for
jessie.
Systems already running Stretch or unstable do not seem to be affected
and can safely use python-lockfile at version 1:0.12.2-2 - this is a
particular problem inside jessie-backports. However, this does mean
that until this bug is fixed, users running Stretch or unstable should
build their own developer packages instead of using the LAVA
repositories on images.validation.linaro.org. We currently do not have
capacity to build separate packages for jessie and stretch.
(LAVA does not recommend using Stretch or unstable for production
systems until Stretch has been released and stretch-backports becomes
available.)
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Reminder about the LAVA repositories
====================================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
2017.2 release
==============
More updates to the jinja2 templates in this release and some new templates.
diffstat output:
lava_scheduler_app/tests/device-types/apq8016-sbc-uboot.jinja2 | 3
lava_scheduler_app/tests/device-types/arduino101.jinja2 | 5
lava_scheduler_app/tests/device-types/b2260.jinja2 | 8 -
lava_scheduler_app/tests/device-types/base-uboot.jinja2 | 26 ++--
lava_scheduler_app/tests/device-types/base.jinja2 | 11 +-
lava_scheduler_app/tests/device-types/d02.jinja2 | 4
lava_scheduler_app/tests/device-types/d03.jinja2 | 1
lava_scheduler_app/tests/device-types/frdm-k64f.jinja2 | 30 +++++
lava_scheduler_app/tests/device-types/hi6220-hikey.jinja2 | 7 -
lava_scheduler_app/tests/device-types/highbank.jinja2 | 2
lava_scheduler_app/tests/device-types/juno-uboot.jinja2 | 1
lava_scheduler_app/tests/device-types/mustang-uefi.jinja2 | 3
lava_scheduler_app/tests/device-types/nexus10.jinja2 | 1
lava_scheduler_app/tests/device-types/nexus4.jinja2 | 1
lava_scheduler_app/tests/device-types/nexus9.jinja2 | 1
lava_scheduler_app/tests/device-types/nrf52-nitrogen.jinja2 | 9 +
lava_scheduler_app/tests/device-types/nucleo-l476rg.jinja2 | 23 ++++
lava_scheduler_app/tests/device-types/nxp-k64f.jinja2 | 1
lava_scheduler_app/tests/device-types/nxp-ls2088.jinja2 | 12 ++
lava_scheduler_app/tests/device-types/overdrive.jinja2 | 1
lava_scheduler_app/tests/device-types/r8a7791-porter.jinja2 | 11 ++
lava_scheduler_app/tests/device-types/stm32-carbon.jinja2 | 11 +-
lava_scheduler_app/tests/device-types/x15.jinja2 | 55
++++++++++
lava_scheduler_app/tests/device-types/x86.jinja2 | 1
24 files changed, 178 insertions(+), 50 deletions(-)
Changes in the 2017.2 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=266b2e633
Commit messages including a reference to LAVA-NNN relate to projects.linaro.org
append the reference to https://projects.linaro.org/browse/
e.g. https://projects.linaro.org/browse/LAVA-854
lava-server - https://git.linaro.org/lava/lava-server.git/
-----------
9df8b354b Remove whitespace from case names
9354cd585 Uboot parameters fixes
297f27900 Fix documentation for auto_login
64b549e9a Fix group visibility and hidden device types
b211434de Document the use of a list for bug 2870
eb8a7bec3 LAVA-854 - Device type template for x15
ad81f33fb Add device template for renesas r8a7791-porter board
8c4d44ae4 Add device template for nucleo-l476rg
112ec6487 Remove send_char, and fix some broken character_delay settings
8f615cd6b Update reviewers handling
0fe482ffb b2260: remove duplicate ip_args from extra_kernel_args
a72c86732 LAVA-847 protect metadata store IOError
53a3c778e If a text_offset is used, quote it to ensure it is a string.
51351e18c Add documentation of the dispatcher-config support
7a1bfb649 Tidy up the Device-type templates for IoT devices
0b5de4dcb Allow lava_scheduler_app to unittest logs to stdout
bc807def4 Fix sizelimit warning handling for V2
13b691615 Summarise job timeouts with example and notes
b5c2ea1a7 Improve notes on portability.
65ae7d644 Fix internal server error due to unpacking TestJobUser.
83e8fa075 Migration doc improvements
9eb145050 Add a unit test for the ethaddr support in b2260
91eb07a36 Creating and restoring backups for V2
479a6295f Add support for setting ethaddr
52b5cf06d Add triage documentation for power failures
526a6e298 LAVA-840 - Remove documentation for USB_DEVICE_WAIT
3415d5319 Allow .py files in examples/sources to be packaged
be33389bf Allow notifications for jobs in Running status.
4d6afd607 Fix blacklisted bug in notifications.
58b48a8f0 Catch all errors during resubmission.
40b010238 Tag admin clean support must return name
14b15f40b LAVA-734 compatibility docs
dd18c26e1 scheduler: do not cascade deletion for some objects
5415eafa7 Do not create unused TestJobUser
2f3cd8389 Bug 2796. Add submitter username to job_details API.
08cd6c01f Revert "master: set the master identity"
be28c8c7b LAVA-814 - Explore pyudev for usb device wait
9150c9973 LAVA-832: Rework device-types management command
70e81493e LAVA-762 link level in timing to pipeline definition
c276df83f LAVA-728 extend ZMQ example to submit and wait
f6a0456a1 LAVA-731 document remote worker issues
41c67ae3b Add device template for NXP-ls2088
ee8964c7b Remove spurious call to save()
2a79a622d Ensure base_ip_args can be overridden
affa68b23 Add a command to add and list workers
10e6ea970 tokens: allow format as CSV
b0b5a1d30 Remove unused config value LAVA_CONFIG_VERSION
69138ac89 Add a command to manage tokens
e09beb1a4 Remove unused custom commands
905a0f9be Fix multinode job definition page
bb0fdc7a8 job logs: Highlight failing results
e1a125aa8 master: set the master identity
b7cb28e49 Fix device schema and test for arduino template
d411416bf LAVA-707 update result case and metadata views
15fc5c62f LAVA-140 - update running job calculation
9fcf92cfd LAVA-515 restore line numbers to YAML definitions
b7266712b LAVA-824 skip unused device types in running table
de6a141aa Expand docs on namespace
c54a5e262 events: fix documentation
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
---------------
ba779908 testdef: set "set +x" to avoid duplicating SIGNALS
ae2ab424 Enforce a download order to produce predictable pipelines
55a39b92 Remove unnecessary boot action in fastboot x15
81b178fc Add a unit test for db410c pipeline generation.
aa96dd75 LAVA-839 - Integrate X15 into LAVA V2
14acd2e2 Fixup u-boot prompts sanity
829fa380 Support CMSIS-DAP devices that do not reset after being flashed
d09a7a74 Simplify send_char support
031ba8e8 Ensure dispatcher.yaml is available for admins
aa6142dd Detect pdu controlled device using availability of power command.
c54891b9 Properly isolate stdout and stderr in unit tests
2d8b3110 Ensure lava-test-reference is available for release
555b6be0 Skip one test if lxc-info is not installed
fc42bd89 Fix support for persistent NFS
57c96b4b Remove duplicate prompt handling.
b0d39204 LAVA-844 allow string and dict for install: git-repos
76998b75 LAVA-830 Use the configured TFTP directory
1f01283f Partially revert a1658c by using a DEALER socket
9045292b Allow boot commands in the job definition
af34dfcb Extend the timeout unit tests to boot actions
9135c5aa Add support for CMSIS-DAP IoT devices and cleanup PyOCD
96dcc7d7 Only match udev events when device is added
a1658c7e slave: use a ROUTER socket to connect to master
35937826 Fix prospector warnings
e4e2fb97 etc/lava-slave: fix documentation of WARN log level
b45cb005 Remove unsed imports
93a282bf Job: log known error status when leaving
ee404039 lava-dispatch: allow to run without zmq logging
26daca49 lava-dispatch: work around a variable clash
c62e1ee9 LAVA-814 - Explore pyudev for usb device wait
989cd97b LAVA-827 ensure the case name is a valid URL
570c1b1e Do not reset the timeout before each connection
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Reminder about the LAVA repositories
===========================
Until Debian Stretch has been released and the new testing suite
(called buster) is open for new uploads, new production releases of
LAVA will only be available from the
images.validation.linaro.org/production-repo repository.
See https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
See also: https://lists.linaro.org/pipermail/lava-announce/2016-December/000023.html
2017.1 release
===========
This release includes substantial updates to the jinja2 templates. If
you have made local changes to these files, you will be prompted
during installation.
/etc/lava-server/dispatcher-config/device-types/apq8016-sbc-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/arndale.jinja2
/etc/lava-server/dispatcher-config/device-types/b2120h410.jinja2
/etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/base.jinja2
/etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2
/etc/lava-server/dispatcher-config/device-types/cubietruck.jinja2
/etc/lava-server/dispatcher-config/device-types/highbank.jinja2
/etc/lava-server/dispatcher-config/device-types/juno-uboot.jinja2
/etc/lava-server/dispatcher-config/device-types/mustang.jinja2
/etc/lava-server/dispatcher-config/device-types/panda.jinja2
/etc/lava-server/dispatcher-config/device-types/qemu.jinja2
Changes in the 2017.1 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=266b2e633
lava-server - https://git.linaro.org/lava/lava-server.git/
--------------
266b2e633 LAVA-795 - Expose lava-publisher event socket settings.
f4b76e48e Fix ordereddict failure
9d3ae86cc Use dpkg-query which is available on all systems
080c3184f Update the device-type context help page
46a055a3b Fixup some jinja2 templates
cecefbfef Timing: improve the description
5bef1d877 LAVA-821 - add support for lava-test-reference
3b26fb234 LAVA-820 record lava-server version in job metadata
ff71a42a6 Append to existing metadata store results
0c9abf643 Fix RST whitespace typo.
05b96af3c Merge timing page into the job page
edc4431f1 Restore access to chart Action buttons
69709881e Improve the timing page and port to the new log format
a00c02aa8 LAVA-535 Make devicetags case-insensitive
36617b955 LAVA-743 - handling complex test operations
50084f1b5 Omit summary button for V2 definitions.
5c8bf4643 Upgrade the dispatcher v2 parser interface.
22409d543 master: send the dispatcher config to the slave
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
----------------------
64ae9131 Report the bootloader load addresses used
2085ffb8 Skip parsing of kernel messages for ssh connections
00fa2d24 Improve allow_fail fault handling
dce680f3 Simplify the connection session handling
ea18594a Use dpkg-query which is available on all systems
2cc64b70 Reinstate git version_tag in version.py
5c05bdae Improve logging
50236943 LAVA-821 - lava-test-reference support
a118e943 Add missing wait definition in base class
2dc955f6 Allow timeout during Linux Kernel message parsing
0404a419 LAVA-819 - record lava-dispatcher version in logs
542effdf Raise an Infra error when download fail in run()
29a232c1 Fix Download retries crashes
12e5789f monitor: make test a bit faster
a90a5184 Fix computation of action duration
ffd1d176 Fix retry logic (partial revert of 7bbd706d)
36ca3c21 LAVA-818 - errors should set the job as Incomplete
f5ac0328 Skip generation of invalid test_case_id
9e5e56c6 Re-enable support for nose tests
bb89772f Merge and re-use BootloaderCommandOverlay and
BootloaderCommandAction
f3864c2f job: fix cleanup when validation failed
8dd450c5 run_actions: clean error messages and exceptions
ec44db64 Properly compute the timeout
f3058a12 atexit is not needed anymore for cleanup
7bbd706d Improve error handling and reporting
a5e0e678 Fix issue when downloading two files with the same name
0d3773ea Call finalize directly in job.cleanup
af608eb2 Remove unused setter, properties and __call__
f946a21d validate: do not extend action.errors
4504aeb9 slave: use the dispatcher configuration
925019c8 Use the dispatcher config to get the dispatcher ip
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Enable the LAVA repository
====================
2016.12 will be the last chance to update of lava-server and
lava-dispatcher into Debian Stretch before the release as Debian 9.0.
This means that users of jessie-backports should now enable the LAVA
production repository - see
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…https://lists.linaro.org/pipermail/lava-announce/2016-November/000019.html
Until Debian makes the 9.0 release, jessie-backports cannot receive
updates of the LAVA releases in 2017, so all updates will only be
available from the LAVA repositories. Shortly after the release, a new
stretch-backports suite will become available and jessie-backports
will be opened again.
In accordance with the Debian freeze policy, it will take 10 days for
the new uploads to migrate into stretch, not the previous 5, so an
upload to jessie-backports will be made at that time.
Note: this may be an issue for armhf or arm64 workers - if you are
using such workers, let us know so that we can prioritise the changes
required to build lava-dispatcher for those architectures.
Updates from jessie-backports
======================
A few updates are required from jessie-backports with 2016.12: lxc and
python-pyudev
The lxc update ensures that jessie and stretch containers can be
correctly created on jessie.
The pyudev support enables lava-dispatcher to monitor udev activity
for certain operations.
These updates can be applied before installing 2016.12, if desired:
$ sudo apt install -t jessie-backports lxc python-pyudev
lava-tool
======
A new version of lava-tool is also available, including a new way of
handling the authentication tokens and the ability to use shortcuts
(like "local" instead of "http://localhost/RPC2").
https://tracker.debian.org/pkg/lava-toolhttps://tracker.debian.org/news/825902
The new version of lava-tool is available via the LAVA staging and
production repositories and the package available in jessie-backports
will be updated in due course.
Unfortunately, it proved not to be possible to export the existing
authentication secrets into the new backend or to support both
backends simultaneously. All users will need to re-add all their
authentication tokens to use lava-tool after this upgrade. The change
has several benefits, including the ability to list and remove
authentication configurations using the new backend and the removal of
a whole host of problems related to using the old backend in
environments where lava-tool was not running in a full terminal. In
addition, this new backend allows lava-tool to support shortcuts which
prevent problems with repetitively entering long and complex URLs.
On installing the new upgrade, users will see the NEWS message:
https://github.com/Linaro/pkg-lava-tool/blob/master/debian/NEWS
lava-tool has changed the backend for token storage.
This means that existing authentications will not be usable to this
new version. This change has been made to fix persistent problems with
the python-keyring support, including:
* DBus errors when used over SSH
* Interaction with gnome-keyring causing authentication failure
* Inability to list or remove authentications added to the keyring
* Inability to work with other keyring solutions.
The new backend is able to list and remove authentications. The new
support also removes the need for a default password on the user
keyring, so lava-tool will no longer pause waiting for password
entry.
2016.12 release
============
Changes in the 2016.12 release are described below, including the
short git commit hash, include:
Use the git commit hash to go directly to that commit using a URL like
https://git.linaro.org/lava/lava-server.git/commit/?id=b19b9648a
lava-server - https://git.linaro.org/lava/lava-server.git/
-----------
541ba7b0a Enable ssh secondary connections on juno-uboot
2d6bb2fcc Handle errors which cause invalid description data
b19b9648a Fix template_mismatch check to use extends
d7f71eae3 Update ipxe x86 nfs to use base_ip_args
fa6b8b3ab Add a note on release-notes on lava-announce
d007d1819 Fix a crash when a device does not have a device dict
d46d6e4fd Protect against invalid parameter submissions
431fd2fc0 Allow pipeline jobs to be used with XML-RPC job_output
f3bf855ba Add Copy to clipboard functionality to MultiNode
19d814e0c Fix some failing tests due to new namespace changes
86950e985 LAVA-745 - expose device_path for all types
083d67d54 Add device-type template for arduino101
fd99c576e Fix UI submission error.
708a38a23 Add SSH support for overdrive
e838de3ba Update Linaro lab links to actual site
d8e127b74 Allow flash_cmds_order to be set
92eefe4aa Adding extra boot commands for secondary media on Juno
74310353c LAVA-792 - add shorthash to changelog
8f860a1e9 Keep result blocks on a single line with MultiNode
3b1d6654e LAVA-499 - Device type template for dragonboard-410c
5c46ea4ab Provide a default fastboot flash commands order.
a61ab2501 Download results XMLRPC support.
347fb3fd3 Allow to forward the messages to additional sockets
4d0753c3a Allow validate_pipeline devices to filter by device type name.
a859e5ede publisher: improve logging and fix logfile owner
f0002efb1 Fix sphinx indentation error
9f9bd71b9 Correctly expose TestSet data in results
c6f9b1df8 Handle decimal conversion error in test case
1769d8b6b Expand hints on new device-types
eec1ca1d0 master: expect a protocol version with HELLO messages
cd8d109f7 master: add the job id to the log messages
a0ac2d66a Add absolute_url to job_details API.
951ab9f68 master: device configuration is empty for secondary media
c0575503d master: improve error reporting when reading env file
f551daa49 Add notes on unattended upgrades
1d2e25d1d Allow override of nfsroot options, not arguments
edaad5419 Update docs to reflect reality of from: support
09503a482 Manually revert d31a8462: errors are already saved
3516b856d add some notes on packages needed for LXC usage
b5ee513bb Rename 'usb-showup' to 'usb-device-wait'
406832582 Drop misleading hostname from device metadata
6a6ee6f4d Handle change to git.linaro.org frontend
d85f99e1b Fix crash when description.yaml is empty
37a94aafc Fix prospector warnings
080499d18 Remove references to wsgi
267d22302 Paste job definition to clipboard as text and not as HTML.
ef09a9698 add a link to the device tag glossary entry
08bd011a4 Update aliases migration for review change
7010775b6 Move metadata extra content to files
95ed9a3ba Expand notes on debugging test jobs and test cases
6d772bb0d Update for pycodestyle pep8 failures
dd58a1182 LAVA-798 - lookup device-types by alias
104feb593 Enable LXC for existing device-types.
376a80021 LAVA-797 - Document changing USB_SHOW_UP_TIMEOUT in job
3e8dbfeca all_devices() api call to do one big query instead of many small ones.
d27f7a4d5 Fix for Debian reproducible builds.
ac8bb8e2a Fix bug #2596 - No result collected for multinode jobs
7b3adad0a All jobs will have a definition, so show it by default.
1e6992f6b Set boot_character_delay in x86 template
5cc0a4746 Extend x86 unit test to check character_delay support
ad4f8780f Combine interfaces block for juno and vland
bb2ecf716 Allow changing the timeout for u-boot-interrupt
19de59978 Add stm32-carbon jinja2
lava-dispatcher - https://git.linaro.org/lava/lava-dispatcher.git/
---------------
19fd79a9 Update preseed support for namespace changes
ef352abf Update zephyr sample jobs to use LAVA functional test urls
1301258a LAVA-498 - Support dragonboard-410c with lxc
1e0b60d2 Ensure pyudev version is >= 0.21
7d6a8134 Update the DFU boot code for namespace changes
161e8b47 Ensure installer tests share the final connection
567ddf0d remove duplicate call
e49bd971 Fix namespace error in flash_cmds_order retrieval.
cb04549d Set connection for secondary media deployments
dcd75e7b Use correct suffix for secondary media deployments
2a7b386f Declare the original ramdisk as compressed
fd73e937 Support DFU devices
6aa7280c LAVA-504 - Dynamic fastboot partition support in pipeline
9228901a LAVA-809 - Show HiKey kernel boot messages
774c7557 Extend namespace fixes for secondary connections
dc5bec22 Action: Always call super() in validate()
9b983cdf GitRepoAction: do not call validate twice
bd112aab Make namespaces common to all dynamic data operations
2e83e2ca Prevent invalid measurements being reported
1768c181 Fix testlevels for multi-test LXC jobs
0a49da8e Give namespaces to all actions and support simpler code flow
2423a123 slave: send the protocol version with HELLO message
983d3775 Ensure fixupdict is checked in test_case_result
08f79918 slave: writing an empty string to a file is valid
c6f382ef Move cleanup_actions from Action to Job
bc56ca24 Remove unused Action, Pipeline and Job members
1256f31e Run.sh should use set -x
a726be8f Allow install-steps to use the git-repo directly
b0ca553a Create a useful exception message
077aca26 slave: don't crash when lava-dispatch can't start
543ed4af Honor USB device wait timeout and add device before returning
connection
0e07089f Fix problem with duplicate test shells failing
f14747c1 Use dnf instead of yum on Fedora version greater than or equal to 22.
efdaabc1 More fix for usage of LXC with a device.
2d188baa Limit repetitive kernel message log entries
c3455a11 Allow newline line separator to be overriden per bootloader
01a482f6 Fix for Debian reproducible builds.
62c07465 Indicate that character delay is in milliseconds
c1f79680 Remove hardcoded character delay in ipxe
e43552b3 Fixing secondary media deployment
e885c881 device-types: add Renesas r8a7795 Salvator-X board
1374289b device-types: add Amlogic meson-gxbb-odroidc2
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Everyone using LAVA should be aware of V1 and V2 and that V1 is going
to be turned off. The plan for how to do that has been discussed
within the team and within Linaro. The most recent presentation on the
migration can be seen here:
http://connect.linaro.org/resource/las16/las16-503/
(Note: the video will auto-start, be aware.)
The slides are available on slideshare:
http://www.slideshare.net/linaroorg/las16503-lava-v2-migration
The dates for each step in the migration are not finalised but will
occur during 2017, this is advance notice.
The highlights of the migration are:
0: During 2017, regular announcements will be made on this list before
each major change. In addition, the regular release announcements will
include details of the major changes in that release.
1: The first major change will be that a new release of lava-server
and lava-dispatcher will *disable* V1 submissions entirely. Any queued
V1 test jobs will become invalid upon this upgrade,
2: A later release will remove V1 documentation and start removing the
V1 source code.
3: A release will be made which *forcibly deletes V1 test jobs,
bundles, filters and image reports*. This is required to complete the
removal of the V1 source code.
4: The final step of the migration is a release containing no V1
source code at all. Only V2 support will remain. This is essential as
the V1 source code is currently blocking some useful additions to V2
as well as plans for future development. The data cannot remain
accessible without the V1 source code. There is not and can not be any
support for converting V1 data to V2.
Please take this chance to read up on the V2 documentation, including
how Results, Queries and Charts replace the deprecated V1 support from
filters and image reports.
Work has started upstream on making sure the migration runs smoothly,
especially when deleting large numbers of V1 test cases.
Remember: Once the first major step is released, upgrades will stop
running any V1 test jobs. Subsequent releases will include changes
which force the deletion of your V1 test data.
If you want to preserve your V1 data, consider setting up a read-only
reference instance which is then pinned at a particular release of
LAVA to prevent deletion of the V1 data.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
There is a known bug with QEMU 2.7 which is affecting installations of
LAVA using QEMU from jessie-backports.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840853
The problem manifests in the apply-overlay-guest action because QEMU
is unable to process the GuestFS operations due to a missing file
which is in the seabios 1.9 package but not in seabios 1.7 - so V2
QEMU test jobs hang at this point.
If you are using QEMU from jessie (version 1:2.1+dfsg-12+deb8u6) then
there is no problem and nothing needs to be done.
If you are using QEMU from jessie-backports (version
1:2.7+dfsg-3~bpo8+2 currently) then there are two simple remedies.
0: You can find the updated seabios package in the LAVA production
repository. See
https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava…
If you have the production repository already available, you can
simply apt update and apt upgrade to install the new seabios package.
1: If you are not using aarch64 QEMU tests or the QEMU UEFI support
for LuvOS, then the version of qemu in jessie will work just fine. To
downgrade, specify the version string to apt:
$ sudo apt install qemu-system-x86=1:2.1+dfsg-12+deb8u6
qemu-system-common=1:2.1+dfsg-12+deb8u6
It is also worth downgrading other qemu packages at the same time.
Check your current install with:
$ dpkg -l | grep qemu
The relevant bug has been re-opened and a backport of seabios has been
uploaded. An updated backport of qemu has also been requested which
would require the installation of the backported seabios 1.9. So if
this bug is not troubling you immediately, you can simply wait for
seabios 1.9 to be available in jessie-backports and install it.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
