To improve the functional test framework, we're looking at a mechanism
based on docker and this has had a knock-on effect on how we use git tags.
Previously, tags were made against the release branch. To get this new
support working, we've moved the 2018.7 tag from the release branch to the
master branch.
This does have one effect when using the developer build scripts:
dch: fatal error at line 1092:
New version specified (2018.4.post1-382-gf2f1af5d8-1) is less than
the current version number (2018.5.post1-2)! Use -b to force.
This is due to the tag issue. To fix, use git pull --tag:
neil@sylvester:lava (master)$ git pull --tag
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (1/1), done.
>From gitlinaro:lava/lava
t [tag update] 2018.7 -> 2018.7
Already up to date.
Developer builds will then proceed.
Note that we've changed the version syntax slightly too.
lava_2018.7-17-gf2f1af5d8-1
See
https://staging.validation.linaro.org/static/docs/v2/debian.html#local-vers…
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
This release has not been uploaded to Debian due to an ongoing transition
relating to Python3.7 - see https://tracker.debian.org/pkg/lava and
https://release.debian.org/transitions/html/python3.7.html
The release is available via the LAVA repositories.
A docker container for lava-dispatcher (only) is also available:
https://hub.docker.com/r/linaro/lava-dispatcher-production-stretch-amd64/ta…
LAVA changes
============
6978ad899 Revert "Make primary key non-editable"
a6c34e83f Output a validation error for missing commands:
839a14d07 Create but not edit objects in admin interface
d54b73918 Simplify and rationalise log permissions
0892c0929 Skip deployment_data test if simg2img not installed
34cac8588 worker: add a link to the admin page
f4c85dd27 Cosmetics
baf22688e lava-slave: cosmetics
1d660d9bd Add further fixes for table searching
5903343d4 Rationalise deployment_data for Python3
e298f5acd api: log when updating device/worker health
5572ad267 Exclude Retired devices from warning messages
d874e845c LAVA-1381 CLI to copy an existing device
0736d9d13 Add rk3328-rock64 device type
35594371c Device: set default health to HEALTH_MAINTENANCE
7974b5398 slave: allow to kill the process cleanly
664977da8 Fix notification token after 1a13b85e
2b7e5768c Prevent admins for creating some objects
6383230a6 Make primary key non-editable
f770b0455 Add docs on adding devices in offline state
faa650080 Remove lxc_cmd_prefix from Action class
c0356957b Extend .gitignore for .pytest_cache/
2e124e609 Fix crash when loader yaml description
f037069b3 Make handle_testcase local
138f35d75 Fix bzr tests
425fcaefc Address the Meta data directly
bad4d83c8 device-types: add meson-gxm-khadas-vim2
5ca3ec4f4 Fix crash after 210140b0
d5c05e3c5 dt: improve the warning when hc are disabled
0216c20cc Do not crash when description.yaml is not readable
01faa5d41 Port 41747ab2 to android lava-test-runner
7099b83a1 Add a section on metalava
779a3c826 Add coverage default configuration
ec8c147bd device-types: convert tegra124-nyan-big to Depthcharge
60c137be0 Remove unused sources
7ce3b9d1b device-types: Remove no-op block
f4c557554 Add a few more notes on best practices
9bb6ee3ce Fix crash introduced by 792b63ad
ee379d343 Validate timeout format inside test definition
fa46c7d72 Use reverse relations to break some cyclic-imports
07889e548 LAVA-419 remove google analytics
b89a51557 Allow for 'os' to be optional in deploy actions
5b539c6e2 LAVA-952 per dispatcher configuration
3244fd57f master: remove unused options
1a13b85e7 Move notifications to a specific file
ed44797dc Move django signal handlers to the right module
35ff9e2aa test: remove dependency on instance.conf
30e59f1eb Fix log reading when end < start
ede84bb8f Loosen the check on health check visibility
f38471b6e Raise JobError if commands do not exist
3a6e4895a Some pylint fixes
296ba75cc Outline how to relate a test result to source
8d7afd663 LAVA-960 - remove device-type from device config
8d7de532c Update development notes for new unit tests
8865e27cf Drop functions expecting a JSON job definition
ced34d7dd Fix Python3 support for csv in XMLRPC
3adc5434d Allow to download only some part of the logs
a0312f682 Fix visibility in pending_jobs_by_device_type
210140b0f Remove support for the old v2 format (output.txt)
09288494c Fix lava-logs crashes with invalid test cases
a76b3c14c LAVA-1370 Fix device state when removing TestJob
0e5d01ee7 Update the lava-run example in the docs
792b63ad3 LAVA-1368 provide access to the callback data
59d345080 LAVA-1350 document qemu support in buster
b667d371c LAVA-1367 - drop pretty printing of device dictionary
bbca0922f Remove post_only and use require_POST
c7ede8881 LAVA-1365 - drop call to remote JS
8ca4aae3b Fix undefined variables
92d592816 scheduler.get_pipeline_device_config: fix crash
e005377e3 adb: add missing parameter to super().run
7f00e7de7 logs: write log lines before calling TestJob.save
ef9664aad Link device hostname and device type on devicedict page.
35f251612 Fix misplaced continue, where a return was intended.
4f6ae7e78 Add sdm845-mtp fastboot template support
77fab7605 Add partition:0 to db820c device-type flash order.
eaee79db9 Dont send an os.linesep when interrupting grub
d23ee01f8 Log if udev passes a path outside /dev
f53cd7e9c Update the warning about over-simplifying integration
a5bf77f17 Remove references to python2 and use python3
e9367c747 By default, dict.get() default value is None
5704e19a3 Remove support for django before 1.10
305f019d4 Add a warning to over-simplifying the integration
b528871ca Extend test_recovery to check the -bl template
bae4d6cad Remove duplicated or unused imports
28049de48 Move debian_package_(version|arch) to lava_common
40fcc1d18 Silence known pytest warnings
804bf8715 Make return statement consistent
37a137c32 Fix exception formating
c0f800991 reduce is part of functools in python3
d3531269d Add missing super().run call
2be843f25 Remove python2 code
abb90c380 Fix pytest for lava_scheduler_app
6cae769ac remove unused function
a87ad6d51 Test lava_common.Timeout class
fd8d4a2e9 Remove export PS1 from ssh connections support
e824fd392 Add notes on dependencies needed for unit tests
b6d2bd601 Set the shell prompt in all cases after 5d7c2090
9bd270263 Allow for existing LAVA_DB_NAME containing hyphens
ca7179686 dateutil is missing from server requirements
0679aa426 LAVA-1338 Handle STARTTC, ENDTC and TESTCASE
5d7c20906 LAVA-1120 - drop export PS1 lava-test
9b7ec7120 Use yaml.safe_dump whenever possible
a475c825f Fix crash with old job definitions
087d8ad73 LAVA-1359 Fix lava-publisher shutdown
00747fbe8 Add missing requirement for guestfs in lava-dispatcher
7db4042df Update cirun to avoid sudo
ecafe79b5 LAVA-1361 Fix timeout computation for some actions
e11c8c9f9 Fix job timeout name
9b64f68fd Add example jinja2 files to released tarball.
80703dd4b LAVA-1360 - Investigate Charts Add Query autocompletion
bee1d82d3 Fix order of the log lines when validate fails
b1af0e771 LAVA-1357 - drop include: functionality
5cb96b87c Finalize: ignore exception but log the error
f8a2b520c Fix typo in lava-slave Jobs SQLITE
1cf21c2ed Remove unnecessary Python 2 imports
a1278b6df Remove "args" from Action.run()
306e25d83 Add a glossary entry for BMC
d623748e3 Move import at the begining of the module
cce095f89 Check user rights when returning dt template
d4df5f3d6 Separate the server and dispatcher requirements
15519d598 Add missing import
f5d60ba5c timing: fix parsing with some timeouts
24a1293e2 make device and heathcheck_job pages similar
69c13d3c1 Fix crash when canceling on device without worker
141aeb176 Show an error if a device does not have a worker
4e289f0e3 device-types: add sun50i-h5-libretech-all-h3-cc
9bde1032e device-types: add sun8i-h3-libretech-all-h3-cc
3872951be device-types: add sun8i-h2-plus-libretech-all-h3-cc
934de4f20 No need to test that a dir exists with os.makedirs
1411a019d Use random.SystemRandom() when applicable
c43ac7034 Use CLoader when applicable
4039281ea LAVA-1260 - using Jinja2 templating
48874353a Improve indexing of lava-target-storage
b21810690 Set out plan for functional testing
325309ba9 Fix outdated comments relating to V1 support.
e0a3aaefe Use contextlib.suppress when applicable
0bbe4d381 Check tag support for multiple devices per role
c4ebcfc97 Remove bare except
9aaa115e3 Fix build errors in docs due to removed sections.
661cda0b5 Allow ci-run just for server
0c9cacae3 Remove unicode from yaml definition
2c1b3481d Allow server unit tests to run without nfs-kernel-server
eed71633a LAVA-1354 specify the tests directories for pytest
12fa843aa Improve invalid_template check
218eb19e9 Allow for lava-server without lava-dispatcher
a2fbd0c5b Add lava-lxc-mocker version to log output.
4d02ab538 Skip wait if no kernel_start_message
a2a6715f6 Apply DRY to X15 jinja2 template
306bf6bb5 Fix lxc-create command of lava-lxc-mocker to parse options
correctly.
2965e7427 Use '-q' option whenever apt-get is called.
69a96a268 lava/lxc-mocker: lxc-attach support -v (set var)
371790242 Remove django python2 compatibility code
047912932 Remove __future__ import
f6813e535 Remove python2 specific code
ec2e64b57 Use apt-get instead of apt.
bd4d0b736 context_help is a string not a tuple
63bb7c5cc Remove dummy constructor
316268eca Only use "is" when comparing against None
75300f938 Raise an infra error when guestfs fails to start
8a9f75db2 LAVA-1305 - catch errors in vexpress mcc
a53748c34 Revert "NFS defaults: vers=3"
6ea49f7fb LAVA-1349 fix state machine when canceling and hc
830df97a0 skip some tests if rpcinfo is not available
95036757e Fix crash when env-dut.yaml is defined
50e347aca Change spawnwindowsize to spawn_maxread
74e000108 LAVA-1351 fix addldapuser user_search encoding
3cd50f3ab LAVA-1349 Cancelled health check should not repeat
a72ecbf3e LAVA-1347 allow instances to disable admin Delete
3b88c861c Drop unused setup files.
4110a04a9 Add support for just running the clean task
acd3ccf05 Fix idempotency of decompress_file
93dc46bc5 Allow adding static_devices with no children
93bb55ad8 device-type: khadas-vim: use default uboot prompt
26cd076ab device-types: meson-gx-common: interrupt w/ctrl-c
70db0802c Expand the documentation on setup commands
91d99739d NFS defaults: vers=3
ff1af7f6d Fix: Duplicated Wait for USB
7473b94bc LAVA-1342 - improve docs on character delays
60039c38c LAVA-973 - Create a packaging script for permissions and user
934fdb6b9 Fix map_scanned_results() when test status string is invalid
c0382aeab device-types: meson8b-odroidc1: interrupt with ctrl-c
731e8df49 device-types: base: allow interrupt_ctrl_list for uboot
e737ced93 Split the scheduler API Help by sections
d5ef2f9a6 LAVA-909 convert JSON submissions to YAML
7f3ce8960 LAVA-1262 - document script side effects
a37a30f7b Don't utf8 encode the log data twice
ac6245249 LAVA-1333 fix visibility check for health checks
6ec3ba863 LAVA-1305 - catch errors in vexpress mcc
94c0991d8 Only setup a database once when using pytest
8801f8d26 LAVA-1320 - Add unit test for new minus-lxc jobs
332a40862 Allow limiting the pexpect spawn_window_size
e71ed3760 LAVA-1277 Document lavacli
e504d0666 LAVA-1299 - clarify Bad and Maintenance health
41747ab2f Use /dev/kmsg for LAVA test signals, if requested
7c9d1a0f4 LAVA-836 publish events from a test shell
576aeb896 LAVA-939 Integrate cc3220SF board
b7dbb12d9 device-types: Introduce sunxi-common
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
Advance notice of some changes coming in 2018.7
{% set device_type = 'foo' %} is going away
===========================================
The purpose of any item in the device dictionary or device-type template is
to provide information to lava-run to operate the test job. As far as
lava-run is concerned, the device_type is completely meaningless. Device
type is a database object, a component of scheduling and administration.
So in 2018.7, all device-type Jinja2 templates have been updated to move
this to a comment and the schema has been updated to remove the device_type
field.
If you have local Jinja2 templates which use {% set device_type = 'foo' %},
this needs to be changed to:
{# device_type = 'foo' #}
or if your template uses:
device_type: foo
This needs to be removed or commented out:
# device_type: foo
Any mention of device-type in a device dictionary will be ignored.
This also means that deploying 2018.7 will involve changes to most upstream
device-type templates. If you use an admin tool like salt, ansible, puppet
etc. then it is worth making this change ahead of time. In the Harston lab,
we create a code review against the git repo containing the device-type
templates. When the admin is ready to upgrade a particular instance, the
changes from the review are applied before the package upgrade is started.
This prevents dpkg interrrupting the upgrade with prompts about changed
configuration files whilst keeping the device-type templates under version
control.
Database configuration is now upstream
======================================
Previous releases have relied on the Debian packaging to do the work of
setting up the PostgresQL database and managing database migrations during
installation when a database already exists.
We took this upstream so that we could manage the process in Python instead
of shell, partly because the psql commands require some complex quoting
which is very difficult to read and understand in shell. The other
advantage is to have more eyes on the database setup code and to have
formal code review of changes. In due course, it may also be helpful with
the use of LAVA with docker.
Availability
============
2018.7 is likely to be available on or after 24th July 2018. At that point,
Debian will still be handling the migration to Python3.7 which has caused
issues in a range of Python packages with new keywords.
https://tracker.debian.org/pkg/lava
On request, we will not be uploading 2018.7 directly to Debian unstable
until this transition is complete. As far as we can tell at the moment,
there are no changes required within the LAVA codebase but there are a lot
of Python dependencies which are yet to be fixed. This will also delay the
arrival of 2018.7 into buster and stretch-backports in Debian.
2018.7 will be available from the LAVA repositories on
images.validation.linaro.org once the release is ready. I'll make a
separate announcement with the full list of changes at that time.
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/
On Fri, 6 Jul 2018 at 09:41, Yuan, ZhanghuiX <zhanghuix.yuan(a)intel.com>
wrote:
> Hi LAVA mainters,
>
>
>
> Nice to meet you.
>
> I want to add my pub key after LAVA installed container.
>
Unless you configure LXC to allow this (and that is outside the scope of
this list), there is no remote access to the container. Adding the public
key will not make it possible to SSH into the container. Hacking sessions
are not supported in the LXC.
Think of the LXC as a transparent layer on the worker. It is not a separate
virtual machine.
What are you trying to achieve?
If you want to access the rootfs of an LXC, you can do so by logging in to
the worker and using the standard LXC support.
If you want to push files from the LXC to a remote location, you can use
the publishing support:
https://staging.validation.linaro.org/static/docs/v2/publishing-artifacts.h…
This is not a LAVA constraint, this is a reality of using LXC.
> But I don’t which parameters should I need add in my job xx.yaml file.
>
> Could you give me some help and advice.
>
>
>
>
>
> >>>>>>>>>>>>>>> Some part information of my xx.yaml file:
> <<<<<<<<<<<<<<<<<<<<<<<<<<
>
>
>
> actions:
>
> - deploy:
>
> timeout:
>
> minutes: 300
>
> to: lxc
>
> os: ubuntu
>
> packages: [python, wget, unzip, python-pexpect, python-serial,
> openssh-server]
>
>
>
> - boot:
>
> prompts:
>
> - '[root(a)(.*) /]#'
>
> timeout:
>
> minutes: 300
>
> method: lxc
>
>
>
> - test:
>
> timeout:
>
> minutes: 300
>
> gefinitionsg
>
> - run:
>
> steps:
>
> lava-test-shell: echo "-------------------------------------
> debug 1 -----------------------"
>
> lava-test-shell: wget --no-proxy -q
> http://otcpkt.bj.intel.com/downloads/pubkey/sys_oak.zip
>
> lava-test-shell: mkdir -p ~/.ssh
>
> lava-test-shell: unzip sys_oak.zip -d ~/.ssh
>
> definitions:
>
> - repository: ssh://
> sys_oak@git-amr-4.devtools.intel.com:29418/pk_osi_test-source.git
>
> from: git
>
> path: <device>/<test_type>_lxc.yaml
>
> name: <test_type>-<device>
>
> params:
>
> BUILD_NUMBER: <build_number>
>
> IMAGE_URL: <image_url>
>
> PRODUCT: <build_name>
>
> IRC_USER: "sys_oak"
>
> PUB_KEY: "ssh-rsa
> AAAAB3NzaC1yc2EAAAADAQABAAABAQDCDJvT5UPE***************************
>
>
>
> >>>>>>>>>>>>>>> Some part information of my xx.yaml file:
> <<<<<<<<<<<<<<<<<<<<<<<<<<
>
>
>
>
>
>
>
>
>
>
>
> Best Regards
>
> Zhanghui Yuan
>
> OTC Production Kernel Integration Test
> _______________________________________________
> Lava-users mailing list
> Lava-users(a)lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lava-users
>
--
Neil Williams
=============
neil.williams(a)linaro.org
http://www.linux.codehelp.co.uk/