lava-announce July 2018

lava-announce@lists.lavasoftware.org

1 participants
4 discussions

by Neil Williams

To improve the functional test framework, we're looking at a mechanism based on docker and this has had a knock-on effect on how we use git tags. Previously, tags were made against the release branch. To get this new support working, we've moved the 2018.7 tag from the release branch to the master branch. This does have one effect when using the developer build scripts: dch: fatal error at line 1092: New version specified (2018.4.post1-382-gf2f1af5d8-1) is less than the current version number (2018.5.post1-2)! Use -b to force. This is due to the tag issue. To fix, use git pull --tag: neil@sylvester:lava (master)$ git pull --tag remote: Enumerating objects: 1, done. remote: Counting objects: 100% (1/1), done. remote: Total 1 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (1/1), done. >From gitlinaro:lava/lava t [tag update] 2018.7 -> 2018.7 Already up to date. Developer builds will then proceed. Note that we've changed the version syntax slightly too. lava_2018.7-17-gf2f1af5d8-1 See https://staging.validation.linaro.org/static/docs/v2/debian.html#local-vers… -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

6 years, 4 months

LAVA 2018.7 production release

by Neil Williams

This release has not been uploaded to Debian due to an ongoing transition relating to Python3.7 - see https://tracker.debian.org/pkg/lava and https://release.debian.org/transitions/html/python3.7.html The release is available via the LAVA repositories. A docker container for lava-dispatcher (only) is also available: https://hub.docker.com/r/linaro/lava-dispatcher-production-stretch-amd64/ta… LAVA changes ============ 6978ad899 Revert "Make primary key non-editable" a6c34e83f Output a validation error for missing commands: 839a14d07 Create but not edit objects in admin interface d54b73918 Simplify and rationalise log permissions 0892c0929 Skip deployment_data test if simg2img not installed 34cac8588 worker: add a link to the admin page f4c85dd27 Cosmetics baf22688e lava-slave: cosmetics 1d660d9bd Add further fixes for table searching 5903343d4 Rationalise deployment_data for Python3 e298f5acd api: log when updating device/worker health 5572ad267 Exclude Retired devices from warning messages d874e845c LAVA-1381 CLI to copy an existing device 0736d9d13 Add rk3328-rock64 device type 35594371c Device: set default health to HEALTH_MAINTENANCE 7974b5398 slave: allow to kill the process cleanly 664977da8 Fix notification token after 1a13b85e 2b7e5768c Prevent admins for creating some objects 6383230a6 Make primary key non-editable f770b0455 Add docs on adding devices in offline state faa650080 Remove lxc_cmd_prefix from Action class c0356957b Extend .gitignore for .pytest_cache/ 2e124e609 Fix crash when loader yaml description f037069b3 Make handle_testcase local 138f35d75 Fix bzr tests 425fcaefc Address the Meta data directly bad4d83c8 device-types: add meson-gxm-khadas-vim2 5ca3ec4f4 Fix crash after 210140b0 d5c05e3c5 dt: improve the warning when hc are disabled 0216c20cc Do not crash when description.yaml is not readable 01faa5d41 Port 41747ab2 to android lava-test-runner 7099b83a1 Add a section on metalava 779a3c826 Add coverage default configuration ec8c147bd device-types: convert tegra124-nyan-big to Depthcharge 60c137be0 Remove unused sources 7ce3b9d1b device-types: Remove no-op block f4c557554 Add a few more notes on best practices 9bb6ee3ce Fix crash introduced by 792b63ad ee379d343 Validate timeout format inside test definition fa46c7d72 Use reverse relations to break some cyclic-imports 07889e548 LAVA-419 remove google analytics b89a51557 Allow for 'os' to be optional in deploy actions 5b539c6e2 LAVA-952 per dispatcher configuration 3244fd57f master: remove unused options 1a13b85e7 Move notifications to a specific file ed44797dc Move django signal handlers to the right module 35ff9e2aa test: remove dependency on instance.conf 30e59f1eb Fix log reading when end < start ede84bb8f Loosen the check on health check visibility f38471b6e Raise JobError if commands do not exist 3a6e4895a Some pylint fixes 296ba75cc Outline how to relate a test result to source 8d7afd663 LAVA-960 - remove device-type from device config 8d7de532c Update development notes for new unit tests 8865e27cf Drop functions expecting a JSON job definition ced34d7dd Fix Python3 support for csv in XMLRPC 3adc5434d Allow to download only some part of the logs a0312f682 Fix visibility in pending_jobs_by_device_type 210140b0f Remove support for the old v2 format (output.txt) 09288494c Fix lava-logs crashes with invalid test cases a76b3c14c LAVA-1370 Fix device state when removing TestJob 0e5d01ee7 Update the lava-run example in the docs 792b63ad3 LAVA-1368 provide access to the callback data 59d345080 LAVA-1350 document qemu support in buster b667d371c LAVA-1367 - drop pretty printing of device dictionary bbca0922f Remove post_only and use require_POST c7ede8881 LAVA-1365 - drop call to remote JS 8ca4aae3b Fix undefined variables 92d592816 scheduler.get_pipeline_device_config: fix crash e005377e3 adb: add missing parameter to super().run 7f00e7de7 logs: write log lines before calling TestJob.save ef9664aad Link device hostname and device type on devicedict page. 35f251612 Fix misplaced continue, where a return was intended. 4f6ae7e78 Add sdm845-mtp fastboot template support 77fab7605 Add partition:0 to db820c device-type flash order. eaee79db9 Dont send an os.linesep when interrupting grub d23ee01f8 Log if udev passes a path outside /dev f53cd7e9c Update the warning about over-simplifying integration a5bf77f17 Remove references to python2 and use python3 e9367c747 By default, dict.get() default value is None 5704e19a3 Remove support for django before 1.10 305f019d4 Add a warning to over-simplifying the integration b528871ca Extend test_recovery to check the -bl template bae4d6cad Remove duplicated or unused imports 28049de48 Move debian_package_(version|arch) to lava_common 40fcc1d18 Silence known pytest warnings 804bf8715 Make return statement consistent 37a137c32 Fix exception formating c0f800991 reduce is part of functools in python3 d3531269d Add missing super().run call 2be843f25 Remove python2 code abb90c380 Fix pytest for lava_scheduler_app 6cae769ac remove unused function a87ad6d51 Test lava_common.Timeout class fd8d4a2e9 Remove export PS1 from ssh connections support e824fd392 Add notes on dependencies needed for unit tests b6d2bd601 Set the shell prompt in all cases after 5d7c2090 9bd270263 Allow for existing LAVA_DB_NAME containing hyphens ca7179686 dateutil is missing from server requirements 0679aa426 LAVA-1338 Handle STARTTC, ENDTC and TESTCASE 5d7c20906 LAVA-1120 - drop export PS1 lava-test 9b7ec7120 Use yaml.safe_dump whenever possible a475c825f Fix crash with old job definitions 087d8ad73 LAVA-1359 Fix lava-publisher shutdown 00747fbe8 Add missing requirement for guestfs in lava-dispatcher 7db4042df Update cirun to avoid sudo ecafe79b5 LAVA-1361 Fix timeout computation for some actions e11c8c9f9 Fix job timeout name 9b64f68fd Add example jinja2 files to released tarball. 80703dd4b LAVA-1360 - Investigate Charts Add Query autocompletion bee1d82d3 Fix order of the log lines when validate fails b1af0e771 LAVA-1357 - drop include: functionality 5cb96b87c Finalize: ignore exception but log the error f8a2b520c Fix typo in lava-slave Jobs SQLITE 1cf21c2ed Remove unnecessary Python 2 imports a1278b6df Remove "args" from Action.run() 306e25d83 Add a glossary entry for BMC d623748e3 Move import at the begining of the module cce095f89 Check user rights when returning dt template d4df5f3d6 Separate the server and dispatcher requirements 15519d598 Add missing import f5d60ba5c timing: fix parsing with some timeouts 24a1293e2 make device and heathcheck_job pages similar 69c13d3c1 Fix crash when canceling on device without worker 141aeb176 Show an error if a device does not have a worker 4e289f0e3 device-types: add sun50i-h5-libretech-all-h3-cc 9bde1032e device-types: add sun8i-h3-libretech-all-h3-cc 3872951be device-types: add sun8i-h2-plus-libretech-all-h3-cc 934de4f20 No need to test that a dir exists with os.makedirs 1411a019d Use random.SystemRandom() when applicable c43ac7034 Use CLoader when applicable 4039281ea LAVA-1260 - using Jinja2 templating 48874353a Improve indexing of lava-target-storage b21810690 Set out plan for functional testing 325309ba9 Fix outdated comments relating to V1 support. e0a3aaefe Use contextlib.suppress when applicable 0bbe4d381 Check tag support for multiple devices per role c4ebcfc97 Remove bare except 9aaa115e3 Fix build errors in docs due to removed sections. 661cda0b5 Allow ci-run just for server 0c9cacae3 Remove unicode from yaml definition 2c1b3481d Allow server unit tests to run without nfs-kernel-server eed71633a LAVA-1354 specify the tests directories for pytest 12fa843aa Improve invalid_template check 218eb19e9 Allow for lava-server without lava-dispatcher a2fbd0c5b Add lava-lxc-mocker version to log output. 4d02ab538 Skip wait if no kernel_start_message a2a6715f6 Apply DRY to X15 jinja2 template 306bf6bb5 Fix lxc-create command of lava-lxc-mocker to parse options correctly. 2965e7427 Use '-q' option whenever apt-get is called. 69a96a268 lava/lxc-mocker: lxc-attach support -v (set var) 371790242 Remove django python2 compatibility code 047912932 Remove __future__ import f6813e535 Remove python2 specific code ec2e64b57 Use apt-get instead of apt. bd4d0b736 context_help is a string not a tuple 63bb7c5cc Remove dummy constructor 316268eca Only use "is" when comparing against None 75300f938 Raise an infra error when guestfs fails to start 8a9f75db2 LAVA-1305 - catch errors in vexpress mcc a53748c34 Revert "NFS defaults: vers=3" 6ea49f7fb LAVA-1349 fix state machine when canceling and hc 830df97a0 skip some tests if rpcinfo is not available 95036757e Fix crash when env-dut.yaml is defined 50e347aca Change spawnwindowsize to spawn_maxread 74e000108 LAVA-1351 fix addldapuser user_search encoding 3cd50f3ab LAVA-1349 Cancelled health check should not repeat a72ecbf3e LAVA-1347 allow instances to disable admin Delete 3b88c861c Drop unused setup files. 4110a04a9 Add support for just running the clean task acd3ccf05 Fix idempotency of decompress_file 93dc46bc5 Allow adding static_devices with no children 93bb55ad8 device-type: khadas-vim: use default uboot prompt 26cd076ab device-types: meson-gx-common: interrupt w/ctrl-c 70db0802c Expand the documentation on setup commands 91d99739d NFS defaults: vers=3 ff1af7f6d Fix: Duplicated Wait for USB 7473b94bc LAVA-1342 - improve docs on character delays 60039c38c LAVA-973 - Create a packaging script for permissions and user 934fdb6b9 Fix map_scanned_results() when test status string is invalid c0382aeab device-types: meson8b-odroidc1: interrupt with ctrl-c 731e8df49 device-types: base: allow interrupt_ctrl_list for uboot e737ced93 Split the scheduler API Help by sections d5ef2f9a6 LAVA-909 convert JSON submissions to YAML 7f3ce8960 LAVA-1262 - document script side effects a37a30f7b Don't utf8 encode the log data twice ac6245249 LAVA-1333 fix visibility check for health checks 6ec3ba863 LAVA-1305 - catch errors in vexpress mcc 94c0991d8 Only setup a database once when using pytest 8801f8d26 LAVA-1320 - Add unit test for new minus-lxc jobs 332a40862 Allow limiting the pexpect spawn_window_size e71ed3760 LAVA-1277 Document lavacli e504d0666 LAVA-1299 - clarify Bad and Maintenance health 41747ab2f Use /dev/kmsg for LAVA test signals, if requested 7c9d1a0f4 LAVA-836 publish events from a test shell 576aeb896 LAVA-939 Integrate cc3220SF board b7dbb12d9 device-types: Introduce sunxi-common -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

6 years, 5 months

Upcoming changes in 2018.7

by Neil Williams

Advance notice of some changes coming in 2018.7 {% set device_type = 'foo' %} is going away =========================================== The purpose of any item in the device dictionary or device-type template is to provide information to lava-run to operate the test job. As far as lava-run is concerned, the device_type is completely meaningless. Device type is a database object, a component of scheduling and administration. So in 2018.7, all device-type Jinja2 templates have been updated to move this to a comment and the schema has been updated to remove the device_type field. If you have local Jinja2 templates which use {% set device_type = 'foo' %}, this needs to be changed to: {# device_type = 'foo' #} or if your template uses: device_type: foo This needs to be removed or commented out: # device_type: foo Any mention of device-type in a device dictionary will be ignored. This also means that deploying 2018.7 will involve changes to most upstream device-type templates. If you use an admin tool like salt, ansible, puppet etc. then it is worth making this change ahead of time. In the Harston lab, we create a code review against the git repo containing the device-type templates. When the admin is ready to upgrade a particular instance, the changes from the review are applied before the package upgrade is started. This prevents dpkg interrrupting the upgrade with prompts about changed configuration files whilst keeping the device-type templates under version control. Database configuration is now upstream ====================================== Previous releases have relied on the Debian packaging to do the work of setting up the PostgresQL database and managing database migrations during installation when a database already exists. We took this upstream so that we could manage the process in Python instead of shell, partly because the psql commands require some complex quoting which is very difficult to read and understand in shell. The other advantage is to have more eyes on the database setup code and to have formal code review of changes. In due course, it may also be helpful with the use of LAVA with docker. Availability ============ 2018.7 is likely to be available on or after 24th July 2018. At that point, Debian will still be handling the migration to Python3.7 which has caused issues in a range of Python packages with new keywords. https://tracker.debian.org/pkg/lava On request, we will not be uploading 2018.7 directly to Debian unstable until this transition is complete. As far as we can tell at the moment, there are no changes required within the LAVA codebase but there are a lot of Python dependencies which are yet to be fixed. This will also delay the arrival of 2018.7 into buster and stretch-backports in Debian. 2018.7 will be available from the LAVA repositories on images.validation.linaro.org once the release is ready. I'll make a separate announcement with the full list of changes at that time. -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

6 years, 5 months

Re: [Lava-announce] [Lava-users] How to add pub key to download source code after LAVA create the container

by Neil Williams

On Fri, 6 Jul 2018 at 09:41, Yuan, ZhanghuiX <zhanghuix.yuan(a)intel.com> wrote: > Hi LAVA mainters, > > > > Nice to meet you. > > I want to add my pub key after LAVA installed container. > Unless you configure LXC to allow this (and that is outside the scope of this list), there is no remote access to the container. Adding the public key will not make it possible to SSH into the container. Hacking sessions are not supported in the LXC. Think of the LXC as a transparent layer on the worker. It is not a separate virtual machine. What are you trying to achieve? If you want to access the rootfs of an LXC, you can do so by logging in to the worker and using the standard LXC support. If you want to push files from the LXC to a remote location, you can use the publishing support: https://staging.validation.linaro.org/static/docs/v2/publishing-artifacts.h… This is not a LAVA constraint, this is a reality of using LXC. > But I don’t which parameters should I need add in my job xx.yaml file. > > Could you give me some help and advice. > > > > > > >>>>>>>>>>>>>>> Some part information of my xx.yaml file: > <<<<<<<<<<<<<<<<<<<<<<<<<< > > > > actions: > > - deploy: > > timeout: > > minutes: 300 > > to: lxc > > os: ubuntu > > packages: [python, wget, unzip, python-pexpect, python-serial, > openssh-server] > > > > - boot: > > prompts: > > - '[root(a)(.*) /]#' > > timeout: > > minutes: 300 > > method: lxc > > > > - test: > > timeout: > > minutes: 300 > > gefinitionsg > > - run: > > steps: > > lava-test-shell: echo "------------------------------------- > debug 1 -----------------------" > > lava-test-shell: wget --no-proxy -q > http://otcpkt.bj.intel.com/downloads/pubkey/sys_oak.zip > > lava-test-shell: mkdir -p ~/.ssh > > lava-test-shell: unzip sys_oak.zip -d ~/.ssh > > definitions: > > - repository: ssh:// > sys_oak@git-amr-4.devtools.intel.com:29418/pk_osi_test-source.git > > from: git > > path: <device>/<test_type>_lxc.yaml > > name: <test_type>-<device> > > params: > > BUILD_NUMBER: <build_number> > > IMAGE_URL: <image_url> > > PRODUCT: <build_name> > > IRC_USER: "sys_oak" > > PUB_KEY: "ssh-rsa > AAAAB3NzaC1yc2EAAAADAQABAAABAQDCDJvT5UPE*************************** > > > > >>>>>>>>>>>>>>> Some part information of my xx.yaml file: > <<<<<<<<<<<<<<<<<<<<<<<<<< > > > > > > > > > > > > Best Regards > > Zhanghui Yuan > > OTC Production Kernel Integration Test > _______________________________________________ > Lava-users mailing list > Lava-users(a)lists.linaro.org > https://lists.linaro.org/mailman/listinfo/lava-users > -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

6 years, 5 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-announce July 2018