lava-users

lava-users@lists.lavasoftware.org

1 participants
912 discussions

LAVA 2017.11 release

by Neil Williams

See https://lists.linaro.org/pipermail/lava-announce/2017-September/000037.html 2017.11 is the second release on the roadmap to the removal of V1. This is the single largest change ever made to the LAVA packages. * All dashboard URLs are permanently disabled in lava-server. * All devices which were not enabled for V2 are now hidden and unusable. * All V1 code has been removed from lava-dispatcher. * All V1 documentation has been removed from lava-server-doc. 2017.11 disables all access to any V1 test data - including V1 only devices. The data itself remains in the database but is not accessible via any API. If you have chosen to keep an archive of V1 test data and have not yet done so, you should do this before installing this upgrade. At the very least, you **must** have a backup of your postgresql database **before** you upgraded to 2017.11 in order to make a V1 test data archive later. 2017.11 requires that **all** workers and each associated master are upgraded to 2017.11 for test jobs to continue running. This is because there have been very large changes to the directories and file layout within both lava-server and lava-dispatcher associated with the removal of the V1 components. The LAVA software team have been checking ahead for the final stage of the roadmap and plans are on track, as described. http:// and https:// changes in django ====================================== A reminder, if you are accessing an instance using http:// (whether localhost or not), then the changes described in the documentation **must** be applied for the instance to operate correctly. This is a security change in Django, not a change in LAVA. https://staging.validation.linaro.org/static/docs/v2/pipeline-debug.html#us… Symptoms include HTTP500 errors but can also include the inability to login to an instance using LDAP or local django accounts, depending on your local apache configuration. Changes in 2017.11 ================== Changes in the 2017.11 release are described below, including the short git commit hash. Use the git commit hash to go directly to that commit using a URL like https://git.linaro.org/lava/lava-server.git/commit/?id=b19b9648a for lava-server or https://git.linaro.org/lava/lava-dispatcher.git/commit/?id=1451e3a4 for lava-dispatcher. Where the changelog message includes a story id like LAVA-552, that id can be added to the base URL: https://projects.linaro.org/browse/ to find out more about the change itself. e.g. https://projects.linaro.org/browse/LAVA-552 All LAVA stories are publicly available without requiring a login. 2017.11 also includes large changes to the packaging of both lava-server and lava-dispatcher. The prompts formerly used to configure a V1 remote worker have been removed. Packaging is maintained at https://github.com/Linaro/pkg-lava-server and https://github.com/Linaro/pkg-lava-dispatcher Packages for 2017.11 have been uploaded to Debian unstable and also the LAVA repositories (for jessie-backports and stretch-backports). lava-server =========== a799fc6c Fix links back to the main instance 5b9656b6 Fix broken plural in header b21f8c8f Fix crash in job_section_log c9526f01 Document the LOG_SIZE_LIMIT setting in settings.conf 63b57c94 Remove lava-mount-masterfs 75b403f4 Remove add_device script 8ddea823 Add description about the disc space requirement for pg_upgradecluster. 5c9dceb2 Remove TemporaryDevice imports 9b21ea48 Removed unused Worker functions 2accb4c3 Matching change to templates for directory change 1d384c4a Update docs regarding device-dictionary lava-server manage command. 32472fac Port 77fd10e2 to scheduler.devices.get_dictionary 114bbfa5 LAVA-1093 - Add a management command to drop all materialized views 8bc9d83a Update dashboard XML-RPC API help for api_version bb090e96 LAVA-552 - add system.api_version support 77fd10e2 LAVA-1092 support passing a context to device_config de7ad3aa Remove dashboard_app views and static files. 4bc68531 Remove templates from dashboard_app. 1a618c27 Remove unused functions d50deca7 api: limit the number of jobs returned 59880e14 Fix HTTP 500 on devicetype health history a39fecf5 Remove unused resources (css, js and images) a32ac3b6 Rename base-bootstrap and content-bootstrap 7c108de1 job: print lava.job result in case of failure b51c419f Remove old templates a9139c1f Use bootstrap template 7915f18f Remove unused resources 7331901a scheduler.api.jobs.show: add failure_comment 71233bff Fix print syntax for python3 compatibility 936d5eca Remove unused tags 4ebb2a39 Remove unused sources 21a7eb18 Save more sql queries by caching values 4d0d867e Remove non pipeline jobs support a6648dbd Retain docs on disabling V1 workers 8eac0d74 Revert "LAVA-950 set the master identity" 68573b1a Remove V1 dependencies a8151559 timing: handle new start/end line format 26cc3ad0 Remove unnecessary js lib beautify. c44009fa Simplify job view e1da85d4 Fix loading of description.yaml 7b41f7ef Add documentation for secondary media writer parameters 1b4d3dd5 LAVA-1081 migrate instance name to settings f3dc1987 Fix up typos from earlier changes faa24f59 LAVA-1079 - remove HIDE_V1 and HIDE_V2 doc options 942b84c3 Remove unused functions and imports e61a4831 LAVA-1056 Drop V1 documentation 126dce0c Fix crash introduced by 42451c9cc f0b28214 Remove v1 codebase 31bb1bf0 Fix notification exception when query is used for data comparison. a6cff700 Remove last references to lava_dispatcher v1 043690d9 Remove references to lava_dispatcher v1 4c9e493c Remove dashboard_app traces from rest of the codebase. c371b8ca drop link to removed page 1b1b638e LAVA-876 - Return empty data stubs for dashboard API calls. 919c10c2 Remove links to dashboard_app in the scheduler_app 1483e9fb Remove leftover from job wizard f7a1c976 docker: allow passing extra options 42451c9c Use all_jobs_with_custom_sort when applicable cc4ffd64 Fix broken links in example test job 0ca5d75c Drop the migration status page. b37fe944 Add a device-type config for the rpi-3b in armv7 32bit mode 9e904a18 Revert "LAVA-876 - Remove access to Dashboard" f07e0950 Fix warning regarding naive datetime e8348425 Remove more references to dashboard_app 103a44be LAVA-876 - Remove access to Dashboard d33c9c98 LAVA-876 - Remove access to Dashboard b6fc46aa Revert "LAVA-1038 add a settings to archive the instance" f75a333d Do not run dashboard test anymore d9163445 Remove references to v1 jobs addb61d9 bug 3268: fix lava-master crash with invalid logs febf91b1 Improve advice on api/help ac8b7df3 fix pep8 error 64025cf3 LAVA-1072 test all template connection syntax 10d5fbe1 Adjust U-Boot load addresses for tegra124 devices to allow bigger kernels f30ef936 LAVA-1065 - Remove dashboard_app urls server wide. 096a2f1c LAVA-1049 - Allow for .yml as well as .yaml for healthchecks 918f0bc7 Fix logic so addldapuser and mergeldapuser work for all LDAP configs 36fcb3c2 Documentation changes for multiple uarts b1de8acf Update instructions for migrating postgres 0ff08373 LAVA-1053 Results limit does not work for queries f9e64aa1 Fix SQL request storm when listing jobs 2d9c4757 No need to save after get_or_create or Create() 8988cb36 LAVA-896 fix level in result export 219fa3dd LAVA-1073 - support for a second UART on a device c934bbbe master: fix database reconnection 4f0e0e62 LAVA-950 set the master identity 316d77fb Expand the device integration guidance. lava-dispatcher =============== d91a5b99 Revert "Stop parsing kernel messages when the end of a panic has been found" 1451e3a4 LAVA-1098 unassigned variable in read_feedback 437d57ba Rationalise the devices directory structure bdffb3bf Add power control to pyocd d9553680 Add feedback check to test shell 06588ddf Receive the udev device directly from the udev rule d811b2a8 Fix license 65d6c6a6 Move arduino101 dfu example urls to images.v.l.o 61cff41c Only log when feedback content was found 0ba35392 Move juno removable URLs to images.validation.linaro.org cb983d97 Cleanup after pytest runs 997265c2 LAVA-1086 add handler to listen to feedback 29a60e38 Add ConfigObj dependency for TFTP check 70bd3fab Fix prospector warnings 6edb8ef9 Fix missing import and other pylint issues 6f7f5f05 Remove lava-lxc-device-* scripts which are unused. 578b054b Use py.test 65c03eca docker: allow passing extra options 7b7d920a device-info should be yaml ef8f7d4c Remove deprecated function 165e14f8 Fix linger period: set it to 10s instead of 5ms 49c500ac Fix double crash when the process crash early 7dbdc2b6 Add test case for secondary deployment writer tool parameters 969459c6 LAVA-1069 - migrate dispatcher device configs 9e8e5258 Use optargs to extend ci-run for python3 e1c1561b Fix python3 breakage in unit test 69277753 Read feedback when finalising connections 0fb44088 LAVA-1067 Refactor fastboot flash operations 9ac71872 Add missing import 94430958 Ensure finalize always finalises protocols 9d712364 Add option to not uniquify secondary image download paths cdfd5909 Fix master cert handling for lava-run and lxc helper 73548f7a Rename dd params to tool for removable media 1802c3e8 Handle master and slave certs as filenames only ed306477 Do not rely on SysV runlevel to ensure container is ready. 75b47c6a Remove loop mounts via guestfs, while applying overlay to sparse image. cd1152e3 Support archlinux and slackware rootfs 6fc024a5 Remove deprecated modules ddf5d2fc Remove v1 code 5561be23 LAVA-1074 - second UART support 758a41a6 Allow configurable writes for secondary storage deployment fb75db4a Add flag to disable uniquifying the download paths -- Neil Williams ============= neil.williams(a)linaro.org http://www.linux.codehelp.co.uk/

7 years, 1 month

403: Forbidden - CSRF verification failed (2017.10-1)

by Dragos Iorgulescu

Hi everyone! I have recently upgraded our production server to 2017.10-1. Whenever anyone tries to login in the WEB UI, either the “403: Forbidden – CSRF verification failed” error is returned, or no error at all (for the latter, the default LAVA page is shown again, but with no login info). I also tried creating a new superuser from the CLI. That worked, but I can’t use it to logon via the WEB UI, so I guess this has nothing to do with the user accounts stored in LAVA. Is this triggered by a change in Django/LAVA? Kind regards, Dragoș

7 years, 1 month

[lava v2] charts and redefining X attribute

by Jonathan Guyot

Hello everybody, I am working with both Jenkins and lava and I thought it was possible to change X attributes in charts to use Jenkins build number instead of a date. In my job, I defined a metadata like this : job_name: quad-ssh-host device_type: ssh *metadata: {jenkins-build: 725, job-context: nightly}* timeouts: job: minutes: 10 action: minutes: 3 connection: seconds: 20 visibility: public […] and I made a query that can select these jobs using job-context as a filter. Then, in the chart, I wanted to use the jenkins-build value to group job instead of a simple date, but the chart does not seem to be able to do that. I made many tries on the lava 2017.2 version. I wonder whether I am misusing lava charts or if it is an issue that need a fix. Thanks in advance for any reply. Jonathan

7 years, 1 month

”502 Proxy Error” for large job log & details (in the WEB UI)

by Dragos Iorgulescu

Hello again! After managing to run a consistent series of tests on one of our boards, I came across a situation where the job log seems to be too big for LAVA to display. When clicking the ”Job details” button, the server eventually sends a ”502 Proxy Error” message. Is there a setting or a workaround to make LAVA serve the job log & details in ”chunks”? I did download the plain log (using, for example, http://LAVA_HOST/scheduler/job/JOB_ID/log_file/plain), and it is, indeed, huge (29 MB), but I was just curious if, maybe a ”log rotation”-like mechanism could be used in these situations (somehow). Thanks in advance! -- /Dragoş

7 years, 1 month

Git error after upgrading to 2017.10-1: "fatal: index-pack failed"

by Dragos Iorgulescu

Hi! Just upgraded from 2017.6 to 2017.10-1. Tried re-running some jobs that worked fine just before the upgrade and noticed that git checkouts can no longer pe performed. The encountered error is: fatal: The remote end hung up unexpectedly\nfatal: early EOF\nfatal: index-pack failed\n Tried checking out manualy on the host, using the same repo. It works. How could I proceed investigating this? Could this be related to some lava user configuration that got reset (I don't know if git-related configs are associated with LAVA)? Thanks in advance (again)! P.S: I have attached the plain job log -- /Dragoş

7 years, 1 month

LAB Notice: Planned maintenance

by Dave Pigott

[bcc’d to everyone to ensure we get full coverage. Please ignore if you have no interest in LAVA and the Lab] Hi all, Starting this week, we are finalising the transition of LAVA main production to LAVA V2. The main update and hopefully minor downtime, will commence at 09:00 UTC on Monday 6th November. Along with this we will also be upgrading all micro-instances to Debian Stretch and LAVA 2017.11. There will be a brief period on Monday morning, which should be less than an hour, when remote logins will be disabled as we upgrade the gateway server. Please note, that when validation.linaro.org <http://validation.linaro.org/> comes back online, LAVA V1 job submissions will be rejected, so if you have any bots which auto-submit V1 jobs, please work with us to transition them to V2. For future reference, for announcements about Lab availability, you should join the lava-users mailing list. Thanks Dave ---------------- Dave Pigott LAVA Lab Lead Linaro Ltd t: (+44) (0) 1223 400063

7 years, 1 month

Very long output line seems to interfere with board output/job run

by Dragos Iorgulescu

Good morning everyone! While running some tests using our Linux distro and a SuperMicro Intel D1521 board, we encounter a situation where a very long line (shown as a "broken line" in LAVA's logs - in the WEB UI) seems to interfere with the job run, in a sense that the job eventually ends in a timeout. This does not happen when that particular set of tests is not run and I am wondering if this might be a LAVA issue. -------------------- A bit of context, first. SuperMicro boards "make sure" your board connection options are limited to using SoL (serial-over-lan), IPMI and a serial port (but in a very limited manner). One is not able to do a "typical" image load, since you may not specify hardware addresses or properly interact with the bootloader. This led to an approach which is best illustrated in the attached diagram. For short: LAVA instantiates an LXC container, from where we run a series of scripts to prepare the board (copy some images & files on an already available Linux instance, altering the Grub menu and then rebooting the board, making it start the image we wish to test) and then run remote commands via SSH. So, basically, the LXC container runs each and every command on the board, acting as the middleman. ------------------- Now, we have the following: * https://paste.debian.net/993516/ -> plain log excerpt that contains the very long line I was talking about (please see line no. 90) * https://paste.debian.net/993517/ -> job definition * https://paste.debian.net/993518/ -> test definition Is there a way to further investigate if this type of output is "giving LAVA a hard time"? Is there a setting that limits interpreted output, somehow? May it be that this is a LAVA issue? Kind regards! -- /Dragoş

7 years, 1 month

"No such method: u'results.run_query'" when trying to access queries from a Python script

by Dragos Iorgulescu

Good morning everyone! I am trying to "play around" with user queries in LAVA2. After reading about them (https://validation.linaro.org/static/docs/v2/lava-queries-charts.html?highl… ), I downloaded the provided sample script (https://validation.linaro.org/static/docs/v2/examples/source/query-results.… ), made the required updates (user names, query name, hostname & the such) and gave it a try. The only thing I get stuck on is an XML-RPC lib error: "No such method: u'results.run_query'" (the whole traceback can be seen here: https://paste.debian.net/993833/ ) I have attached the modified script I am trying to run at the moment and a screenshot of the query settings. The API Help page is not reachable/present at the moment (nor on https://validation.linaro.org/static/api/help or on our own LAVA instance ), so I am unable to check whether or not the method's name has changed since the last doc update. What am I missing or doing wrong? Many thanks in advance! Have a great day! -- /Dragoş

7 years, 1 month

PXE Booting and flashing device

by Rees, Kevron

I have a minnowboard turbot device that I want lava to boot, flash, reboot to flashed image, and then run tests. I would like to boot the device via PXE networking to an initramfs where I will flash an image to permanent storage. I can use efibootmgr within the initramfs image to set next-boot to the permanent storage device. Once booted from the permanent storage device, lava tests will run as per normal. It appears that many of the ipxe examples I have found simply run tests on the initramfs itself. Is it possible to flash the device once the initramfs has booted? It seems like it would be simple for lava-dispatcher to run wget http://someurl/foo.img | dd of=/dev/sda once the initramfs has reached login. However, I do not know if there is such a deploy mechanism. Does anything already exist within lava to help achieve this model? Is there a better way? Any feedback would be appreciated. -Kevron

7 years, 1 month

Truncated commands & output in LAVA2

by Dragos Iorgulescu

Hi all! We have been working on integrating a cn8304-based board (uses uboot). We noticed some strange behavior that might be related to LAVA2. On average, 2 out of 3 job runs end up with errors/timeouts. We use the same job definition and device each and every time. Looking through LAVA logs, it seems that commands and some output gets truncated, thus malforming the commands being sent or the prompts that LAVA expects. The prompt we expect is "EBB8304> ", but sometimes it gets truncated to "304>". Also, a command gets mixed up somehow, and instead of "setenv loadinitrd 'tftpboot 0x60000000 358/tftp-deploy-VLCRzw/ramdisk/ramdisk.ext4.gz.uboot'", the board receives "EBB8304> setenv loadkernel 'tftpboot 0x40setenv loadinitrd 'tftpboot 0x60000000 358/tftp-deploy-VLCRzw/ramdisk/ramdisk.ext4.gz.uboot'" Details: * The job definition: https://paste.debian.net/993349/ * An excerpt from the LAVA2 job log/output, showing what gets mixed up: https://paste.debian.net/993350/ * The plain job log is attached to this e-mail Many thanks in advance! -- /Dragoş

7 years, 1 month

← Newer
1
...
74
75
76
77
78
79
80
...
92
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users