lava-users

lava-users@lists.lavasoftware.org

1 participants
930 discussions

Enabling ZMQ authentication leads to lava-logs offline

by Chase Qi

Hi, I have lava-master and lava-slave v2018.1 installed, and a qemu device added. Test job can be scheduler. Then I followed https://validation.linaro.org/static/docs/v2/pipeline-server.html#using-zmq… to enable ZMQ authentication. Certificates were generated correctly, public certificates were copied to master and slave respectively. With the following configs: lava-master ``` MASTER_SOCKET="--master-socket tcp://*:5556" LOGLEVEL="DEBUG" ENCRYPT="--encrypt" MASTER_CERT="--master-cert /etc/lava-dispatcher/certificates.d/master.key_secret" SLAVES_CERTS="--slaves-certs /etc/lava-dispatcher/certificates.d/" ``` lava-slave ``` MASTER_URL="tcp://192.168.11.214:5556" LOGGER_URL="tcp://192.168.11.214:5555" HOSTNAME="--hostname lava-slave1" LOGLEVEL="DEBUG" ENCRYPT="--encrypt" MASTER_CERT="--master-cert /etc/lava-dispatcher/certificates.d/master.key" SLAVE_CERT="--slave-cert /etc/lava-dispatcher/certificates.d/slave1.key_secret" ``` After lava-master and lava-slave restarted, I see the following logs. Seems the connect was established, but lava-logs went offline. lava-master ``` 2018-01-30 11:05:50,260 DEBUG lava-slave1 => PING(20) 2018-01-30 11:05:52,086 DEBUG lava-master => PING(20) 2018-01-30 11:06:08,728 DEBUG lava-logs => PING(20) 2018-01-30 11:06:10,261 INFO scheduling health checks: 2018-01-30 11:06:10,270 DEBUG -> disabled on: lxc, qemu 2018-01-30 11:06:10,271 INFO scheduling jobs: 2018-01-30 11:06:10,272 DEBUG - lxc 2018-01-30 11:06:10,292 DEBUG - qemu 2018-01-30 11:06:10,332 DEBUG lava-slave1 => PING(20) 2018-01-30 11:06:12,115 DEBUG lava-master => PING(20) 2018-01-30 11:06:20,252 INFO [POLL] Received a signal, leaving 2018-01-30 11:06:20,254 INFO [CLOSE] Closing the controler socket and dropping messages 2018-01-30 11:06:21,203 INFO [INIT] Dropping privileges 2018-01-30 11:06:21,204 DEBUG Switching to (lavaserver(114), lavaserver(119)) 2018-01-30 11:06:21,204 INFO [INIT] Marking all workers as offline 2018-01-30 11:06:21,209 INFO [INIT] Starting encryption 2018-01-30 11:06:21,211 DEBUG [INIT] Opening master certificate: /etc/lava-dispatcher/certificates.d/master.key_secret 2018-01-30 11:06:21,238 DEBUG [INIT] Using slaves certificates from: /etc/lava-dispatcher/certificates.d/ 2018-01-30 11:06:21,245 INFO [INIT] LAVA master has started. 2018-01-30 11:06:21,246 INFO [INIT] Using protocol version 2 2018-01-30 11:06:41,247 WARNING lava-logs is offline: can't schedule jobs 2018-01-30 11:07:01,255 WARNING lava-logs is offline: can't schedule jobs 2018-01-30 11:07:04,433 INFO lava-slave1 => HELLO 2018-01-30 11:07:04,433 WARNING New dispatcher <lava-slave1> 2018-01-30 11:07:09,450 DEBUG lava-slave1 => PING(20) 2018-01-30 11:07:21,260 WARNING lava-logs is offline: can't schedule jobs 2018-01-30 11:07:29,477 DEBUG lava-slave1 => PING(20) 2018-01-30 11:07:41,265 WARNING lava-logs is offline: can't schedule jobs ``` lava-slave ``` 2018-01-30 11:06:10,283 DEBUG PING => master (last message 20s ago) 2018-01-30 11:06:10,335 DEBUG master => PONG(20) 2018-01-30 11:06:30,356 DEBUG PING => master (last message 20s ago) 2018-01-30 11:07:04,379 INFO [INIT] LAVA slave has started. 2018-01-30 11:07:04,380 INFO [INIT] Using protocol version 2 2018-01-30 11:07:04,390 INFO [INIT] Starting encryption 2018-01-30 11:07:04,390 DEBUG Opening slave certificate: /etc/lava-dispatcher/certificates.d/slave1.key_secret 2018-01-30 11:07:04,413 DEBUG Opening master certificate: /etc/lava-dispatcher/certificates.d/master.key 2018-01-30 11:07:04,414 INFO [INIT] Connecting to master as <lava-slave1> 2018-01-30 11:07:04,415 INFO [INIT] Greeting the master => 'HELLO' 2018-01-30 11:07:04,440 INFO [INIT] Connection with master established 2018-01-30 11:07:04,442 INFO Master is ONLINE 2018-01-30 11:07:04,443 INFO Waiting for instructions 2018-01-30 11:07:09,450 DEBUG PING => master (last message 5s ago) 2018-01-30 11:07:09,455 DEBUG master => PONG(20) ``` >From django admin console, I see lava-slave1 still is online, but both lava-master and lava-logs workers went offline, and it stopped scheduling test job. Have you guys ever see/hit this issue? Any advice and suggestions would be appreciated. Thanks, Chase

7 years, 6 months

IOZone test to measure performance of DUT

by Florence ROUGER JUNG

Hi All, We use IOZone test to measure performance of our DUT Same command executed several time in a Lava session provide a diversity of results iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp Example1: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5053 4420 12471 12137 102400 16 5070 5218 22101 22207 102400 512 11733 12630 40799 40862 102400 1024 11446 11494 39982 39976 102400 16384 13839 14235 42093 42094 Example 2: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5088 5006 13496 13095 102400 16 5395 5549 17199 17220 102400 512 9203 10038 27819 27586 102400 1024 9382 8482 32514 32430 102400 16384 13569 13986 41992 42081 When same command is executed without LAVA with same DUT, results are more homogeneous (similar to exemple1) and permit to define clear targets How to reduce interaction from Lava to measure performance of DUT ? Is it possible to disable some checks / interactions with DUT during execution of each test (around 3 / 4 minutes) ? Thanks in advance for your answer Florence Rouger-Jung

7 years, 6 months

Using power-on command with pyocd boot method

by Andrei Narkevitch

Hi, our boards are powered via PDU and before flashing with pyocd, they need to be powered up first. I defined power_on_command in the device dictionary, but it is ignored. It looks like an only limited set of methods add ResetDevice action to the pipeline and pyocd is not among them. Contrarily, if defined, power_off command always gets invoked because it is added by FinalizeAction to every test. I'm wondering if there is any way to issue the power-on command without modifying the dispatcher pipeline source code. Thanks, Andrei Narkevitch, Cypress Semiconductor This message and any attachments may contain confidential information from Cypress or its subsidiaries. If it has been received in error, please advise the sender and immediately delete this message.

7 years, 7 months

Request to know one example on prepare-scp-overlay

by Chetan Sharma

Hi Everyone Can anyone share sample Yaml File and Usecase using "prepare-scp-overlay" using MultiNode. Some calls can only be made against specific actions. Specifically, the prepare-scp-overlay action needs the IP address of the host device to be able to copy the LAVA overlay (containing the test definitions) onto the device before connecting using ssh to start the test. This is a complex configuration to write. -- Thanks & Regards Chetan Sharma

7 years, 7 months

Procedure to install lava-coordinator setup

by Chetan Sharma

Hi Everyone, Can anyone share a step by step process to install Lava-coordinator in existing Setup to execute multinode job. Existing Setup :- 1. Lava-Server (Machine 1) 2. Lava-worker (Machine 2) 3. Lava-worker (Machine 3) -- Thanks & Regards Chetan Sharma

7 years, 7 months

lava-tool wait-job-events

by Loys Ollivier

Hello everyone, I am using lava-tool to monitor my jobs. Previously I used: $ lava-tool submit-job --block Using version of lava-tool 0.23 I now have this message: --> This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command. But "wait-for-job" doesn't exist. There is a "wait-job-events" option though. I tried this one and it doesn't return even once the job has finished. If I manually stop it and restart it with the same job number I get as output: --> Job already finished with status Complete. Command I'm using: $ lava-tool wait-job-events --job-id 20 http://user@lava-server Is there anything I'm doing incorrectly ? Or are you aware of this bug ? Thanks ! -- Loys OLLIVIER

7 years, 7 months

Booting with Coreboot/Depthcharge

by Guillaume Tucker

A change was sent a while ago to add support for the Coreboot / Depthcharge bootloader which is used on Chromebook devices. This is useful in particular to avoid having to install U-Boot on Chromebook devices. See this Gerrit review here for previous history: https://review.linaro.org/#/c/15203/ I'm now opening this case again to try and get this resolved, there seem to be several issues with the original patch that would need to be clarified. Also, some things might have changed since then in LAVA or Coreboot which could potentially lead to a different approach - any feedback on this would be welcome. To start with, I understand that running mkimage on the dispatcher is not a valid thing to do, it should receive a FIT (flattened image tree) kernel image ready to be booted. This complicates things a bit for projects like kernelci.org where only a plain kernel image is built and ramdisks are served separately, but it's fair enough to say that LAVA is not meant to be packaging kernel images on the fly. Then I believe creating the command line file in LAVA should be fine, although it probably makes more sense to have both the FIT image and cmdline file generated by the same build system. In any case, both files would need to be served from the dispatcher TFTP server to the target device running Coreboot / Depthcharge. So the idea was basically to have an option in Coreboot / Depthcharge to interactively tell it where to find these files for the current job to run, say: <JOB_NUMBER>/tftp-deploy-<RANDOM>/kernel/vmlinuz <JOB_NUMBER>/tftp-deploy-<RANDOM>/kernel/cmdline It looks like the current patch in Gerrit relies on this location to be hard-coded in the bootloader, which works fine for a private development set-up but not for LAVA. To recap, my understanding is that the "depthcharge" boot support code in LAVA would need to: * maybe create the cmdline file with basically the kernel command line split up with one argument per line * or just download the cmdline file along with the vmlinuz FIT * place both the cmdline and vmlinuz FIT files in the job's TFTP directory on the dispatcher * turn on the device and open the serial console... * interactively pass at least the path to the job TFTP directory on the serial console (and if possible the server IP address as well, and maybe even the individual file names rather than hard-coded vmlinuz and cmdline) * look for a bootloader message to know when the kernel starts to load and hand over to the next action (login...) Please let me know if this sounds reasonable or if we should be doing anything differently. I think it would be good to have some agreement and a clear understanding of how this is going to be implemented before starting to work on the code again. Best wishes, Guillaume

7 years, 7 months

LAB Notice: LAVA Instance downtime

by Dave Pigott

[bcc’d to everyone] Hi all, Next week, starting on Monday, we will be upgrading all the LAVA servers to Debian Stretch and then upgrading to LAVA 2018.01. This will involve minimal downtime for each instance. Devices will be off-lined ahead of each upgrade. The order of upgrade will be: Monday: LNG: lng.validation.linaro.org PMWG: pmwg.validation.linaro.org Tuesday: Production: validation.linaro.org LKFT: lkft.validation.linaro.org All being well, the downtime, i.e. the time you will not be able to submit jobs or see the web site, will be of the order of one hour per instance. If that is going to change I will send out a new notification. Thanks for your patience, Dave ---------------- Dave Pigott LAVA Lab Lead Linaro Ltd t: (+44) (0) 1223 400063

7 years, 7 months

Re: [Lava-users] No result coming when job submitted in lava

by Robert Marshall

Smita, Yes I looked and confirm that analysis, you've not commented on what the U-Boot prompt is? I'm putting lava-users back on the cc list in case others wish to comment, and making my comment below clearer Robert "Gumansingh, Smita" <Smita_Gumansingh(a)mentor.com> writes: > Hi Robert, > > Have you got any chance to see my bbb health check log > > bbb health check fails at this point > > ------------ ---------------------- > send-reboot-commands timed out after 179 seconds > end: 2.4.1.1 send-reboot-commands (duration 00:02:59) > ---------------------------- > > Please have a look on this log https://pastebin.com/gPmMG84J > > Thanks & Regards, > Smita Gumansingh > > ________________________________________ > From: Robert Marshall <robert.marshall(a)codethink.co.uk> > Sent: Wednesday, January 10, 2018 6:21 PM > To: Gumansingh, Smita > Subject: Re: [Lava-users] No result coming when job submitted in lava > > Smita > > Though you only to replace that line *if* the U-Boot prompt doesn't ^ need > consist of "U-Boot⇒" - interrupt the BBB boot to see what it actually is, > and if it is necessary, you also need to remove the other line - > apologies for the instructions not being clear here. > > Thanks for the fuller output in the other email! > > Robert > > "Gumansingh, Smita" <Smita_Gumansingh(a)mentor.com> writes: > >> Thanks Robert for the quick response >> >> Currently u-boot is showing in /etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 is: >> >> {% set bootloader_prompt = bootloader_prompt|default('U-Boot') %} >> >> As you suggested I added the line: {% set bootloader_prompt = bootloader_prompt|default('⇒') %} and submitted a job >> >> No output from the job when submitted... >> >> Thanks & Regards, >> Smita Gumansingh >> >> ________________________________________ >> From: Robert Marshall <robert.marshall(a)codethink.co.uk> >> Sent: Wednesday, January 10, 2018 5:08 PM >> To: Gumansingh, Smita >> Cc: lava-users(a)lists.linaro.org >> Subject: Re: [Lava-users] No result coming when job submitted in lava >> >> Hi, some comments below! >> >> "Gumansingh, Smita" <Smita_Gumansingh(a)mentor.com> writes: >> >>> Hi, >>> I am new to lava and trying to submit a job on lava scheduler ,job submitted nut no result is coming. I am trying to >>> test the CIP Kernel on the Beaglebone Black(board is physically connected to my linux machine). Health checkup is >>> working somehow . >> >> By 'somehow' do you mean the health check is completing correctly and the >> device is shown as online? >> >>> I am following the steps from here >>> https://wiki.linuxfoundation.org/civilinfrastructureplatform/ciptestingrefe… >>> >>> Pre-requise: >>> 1. I have built(cip_v4.4.92) the CIP Kernel with Kernel CI as the steps mentioned in >>> https://wiki.linuxfoundation.org/civilinfrastructureplatform/cipsystembuild… >>> 2.Target is booted and up and flashed with debian 4.9 rootfs >>> >> >> What is the U-Boot prompt with this version, if it is ⇒ rather than U-Boot⇒ >> >> on the vagrant machine you need to >> >> sudo vi /etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 >> And add the line: {% set bootloader_prompt = bootloader_prompt|default('⇒') %} >> >> >>> Job Definination is pasted here >>> >>> https://pastebin.com/YwnPXidK >>> >>> Need help for going further !!!! >> >> Do you get any output from the job? >> >>> >>> Thanks & Regards, >>> Smita Gumansingh >>> >> >> Robert

7 years, 7 months

No result coming when job submitted in lava

by Gumansingh, Smita

Hi, I am new to lava and trying to submit a job on lava scheduler ,job submitted nut no result is coming. I am trying to test the CIP Kernel on the Beaglebone Black(board is physically connected to my linux machine). Health checkup is working somehow . I am following the steps from here https://wiki.linuxfoundation.org/civilinfrastructureplatform/ciptestingrefe… Pre-requise: 1. I have built(cip_v4.4.92) the CIP Kernel with Kernel CI as the steps mentioned in https://wiki.linuxfoundation.org/civilinfrastructureplatform/cipsystembuild… 2.Target is booted and up and flashed with debian 4.9 rootfs Job Definination is pasted here https://pastebin.com/YwnPXidK Need help for going further !!!! Thanks & Regards, Smita Gumansingh

7 years, 7 months

← Newer
1
...
72
73
74
75
76
77
78
...
93
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users