lava-users

lava-users@lists.lavasoftware.org

1 participants
912 discussions

Re: [Lava-users] Why make the decision to limit job context in lava master?

by Larry Shen

Thanks, Remi, So, looking forward to your patch which allow admins to extend the white list with a variable in the settings. Regards, Larry From: Remi Duraffort <remi.duraffort(a)linaro.org> Sent: Thursday, May 23, 2019 3:59 PM To: Larry Shen <larry.shen(a)nxp.com> Cc: lava-users(a)lists.lavasoftware.org Subject: [EXT] Re: [Lava-users] Why make the decision to limit job context in lava master? Caution: EXT Email Hello Larry, 1. We mainly have 2 kinds of situations: a) Something like "uboot_ums_flash" which already in linaro lava tree. In the past, this can be overridden by job context. And I guess a lots of other variables which spread every corner of different jinja2 files. Huge I think, I really worried how lava can add so many keys in whitelist... Why do you have to update uboot_ums_flash in the job definition and not in the device dictionary? This sounds more like a device specific information instead of a job definition one. Anyway, having a long list of keys in the white list is not really a problem. This is only a python array, nothing more. b) Something which just used in device jinja2 to control some different command in different situations. I know linaro accept upstreams for device-type, I have no idea if private lab's device jinja2 also useful for other people. Contributions are welcome. A rule of thumb for device-type integration/templates is often: is the device available for purchase by someone outside your company? If yes, then it's a good idea to upstream it. If not, then it's more a case-by-case decision. 2. Sounds you guys will not rollback this commit because security issue. So two suggestions: Sorry no :( a) I don't know how this security issue could impact for an internal lab which not exposed to external internet. Anyway, if possible to add a configure to any settings to let admin to decide if we care this security issue? More details to come later on, but I don't think this is a good idea. b) If a) not easy to do or not accept, if possible this whitelist be stored in database or other persist file, so user can free to add his own keyword to whitelist, then even we will upgrade lava to later new version, we can still remain the keyword setting in the past, meanwhile user still can free to add any keyword to whitelist without upstream again and again for this hard coded whitelist, I don't think this make sense. I'm currently writing down a patch to allow admins to extend the white list with a variable in the settings. Please consider. BTW, as Tim Jaacks said in other thread: this limit not happen in multinode job, is it a bug, so next release we will also have this backdoor closed? I'm fixing this second issue in https://git.lavasoftware.org/lava/lava/merge_requests/547<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.lavas…> Thanks for reporting it. Rgds -- Rémi Duraffort LAVA Team, Linaro

5 years, 7 months

auto_login and minimal boot

by Diego Russo

Hello, In my job definition I have a minimal boot: https://staging.validation.linaro.org/static/docs/v2/actions-boot.html?high… Documentation says "auto-login and transfer_overlay are both supported for this method." But if I add auto_login the schema validation in the submit page will raise a warning: "Valid definition with warnings: extra keys not allowed @ data['actions']['boot']['minimal']" Of course if I take the auto_login out, the validation is green. If I leave the auto_login, it will be used. I think the validation needs to reflect what documentation says. Cheers -- Diego Russo | Staff Software Engineer | Mbed Linux OS ARM Ltd. CPC1, Capital Park, Cambridge Road, Fulbourn, CB21 5XE, United Kingdom http://www.diegor.co.uk - https://os.mbed.com/linux-os/ IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

5 years, 7 months

Re: [Lava-users] Why make the decision to limit job context in lava master?

by Larry Shen

Thanks, Remi, 1. We mainly have 2 kinds of situations: a) Something like "uboot_ums_flash" which already in linaro lava tree. In the past, this can be overridden by job context. And I guess a lots of other variables which spread every corner of different jinja2 files. Huge I think, I really worried how lava can add so many keys in whitelist... b) Something which just used in device jinja2 to control some different command in different situations. I know linaro accept upstreams for device-type, I have no idea if private lab's device jinja2 also useful for other people. 2. Sounds you guys will not rollback this commit because security issue. So two suggestions: a) I don't know how this security issue could impact for an internal lab which not exposed to external internet. Anyway, if possible to add a configure to any settings to let admin to decide if we care this security issue? b) If a) not easy to do or not accept, if possible this whitelist be stored in database or other persist file, so user can free to add his own keyword to whitelist, then even we will upgrade lava to later new version, we can still remain the keyword setting in the past, meanwhile user still can free to add any keyword to whitelist without upstream again and again for this hard coded whitelist, I don't think this make sense. Please consider. BTW, as Tim Jaacks said in other thread: this limit not happen in multinode job, is it a bug, so next release we will also have this backdoor closed? Regards, Larry -----Original Message----- From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of lava-users-request(a)lists.lavasoftware.org Sent: Tuesday, May 21, 2019 11:13 PM To: lava-users(a)lists.lavasoftware.org Subject: [EXT] Lava-users Digest, Vol 9, Issue 19 Caution: EXT Email Send Lava-users mailing list submissions to lava-users(a)lists.lavasoftware.org To subscribe or unsubscribe via the World Wide Web, visit https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.lav… or, via email, send a message with subject or body 'help' to lava-users-request(a)lists.lavasoftware.org You can reach the person managing the list at lava-users-owner(a)lists.lavasoftware.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Lava-users digest..." Today's Topics: 1. Re: Why make the decision to limit job context in lava master? (Remi Duraffort) 2. Re: timeouts for deploy vs http-download (Remi Duraffort) 3. Re: Using transfer overlay (Pete Dyer) ---------------------------------------------------------------------- Message: 1 Date: Tue, 21 May 2019 17:02:35 +0200 From: Remi Duraffort <remi.duraffort(a)linaro.org> To: lava-users <lava-users(a)lists.lavasoftware.org> Subject: Re: [Lava-users] Why make the decision to limit job context in lava master? Message-ID: <CANJfhHeXiuVi02dMfmzeQ5EjnP=SqDymDNoyfwDB2XNN8ckodg(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello, we had to enforce the content of the context dictionary in order to fix a security issue that we found recently. The full details of the security issue will be disclosed when a CVE is available. We know that this is annoying for many people so we tried to collect all the valid use cases before the previous release. Which variables are you setting in the context? Is the corresponding code upstreamed? Cheers Le lun. 20 mai 2019 à 10:49, cnspring2002 <cnspring2002(a)aliyun.com> a écrit : > +1, we also have same issue, had to pending new version deploy. > > Message: 1 > Date: Mon, 20 May 2019 05:39:40 +0000 > From: Larry Shen <larry.shen(a)nxp.com> > To: "lava-users(a)lists.lavasoftware.org" > <lava-users(a)lists.lavasoftware.org> > Subject: [Lava-users] Why make the decision to limit job context in > lava master? > Message-ID: > < > DBBPR04MB63291E1DC4F5F202DEDFE16699060(a)DBBPR04MB6329.eurprd04.prod.out > look.com > > > > Content-Type: text/plain; charset="utf-8" > > For 2019.04 version, we see next: > > Job context > ----------- > > > The schema validator is now checking the content of the `context` dictionary. Only the following keys are now allowed: > > > * `arch`, `boot_console`, `boot_root`, `cpu`, `extra_options`, > `guestfs_driveid`, `guestfs_interface`, `guestfs_size`, `machine`, > `memory`, `model`, `monitor`, `netdevice`, `serial`, `vga` > > * `bootloader_prompt`, `console_device`, `extra_kernel_args`, > `extra_nfsroot_args`, `kernel_loglevel`, `kernel_start_message`, > `lava_test_results_dir`, `menu_interrupt_prompt`, `mustang_menu_list`, > `test_character_delay`, `tftp_mac_address` > > Jobs using keys that are not listed in this list will be rejected. > > > We usually set an customized context in job, and in device-type jinja2, use this context to just different value to set proper parameters. > After this limit, all things break! > > So, my question is: > > > lava could be designed to as a framework to give freedom to users to do their things as in the past, why we now enhance so many limits to users? > And additional, and workaround for my scenario? > > Regards, > Larry > > > > > _______________________________________________ > Lava-users mailing list > Lava-users(a)lists.lavasoftware.org > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist > s.lavasoftware.org%2Fmailman%2Flistinfo%2Flava-users&data=02%7C01% > 7Clarry.shen%40nxp.com%7C2ebbb071596f41c8816208d6ddfee338%7C686ea1d3bc > 2b4c6fa92cd99c5c301635%7C0%7C0%7C636940484145033600&sdata=cpnWI8y6 > NzawoUy%2BOtunqchIyXsaHO0ryYzPyJf3Gw0%3D&reserved=0 -- Rémi Duraffort LAVA Team, Linaro

5 years, 7 months

Using transfer overlay

by Pete Dyer

Hello, I'm using transfer overlay to get a set of tests onto the target. However I want to untar it into /scratch instead of / because of storage requirements. This I can do quite easily. However the tests are going to try to run from / (e.g. /lava-10473/bin/lava-test-runner /lava-10473/1) Is there a way of running tests from /scratch/lava-10473.... ? Or a way of defining a command to run immediately after the transfer overlay that makes a link between /lava-10473 and /scratch/lava-10473 ? Thanks. Pete IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

5 years, 7 months

Re: [Lava-users] Why make the decision to limit job context in lava master?

by cnspring2002

+1, we also have same issue, had to pending new version deploy. Message: 1 Date: Mon, 20 May 2019 05:39:40 +0000 From: Larry Shen <larry.shen(a)nxp.com> To: "lava-users(a)lists.lavasoftware.org" <lava-users(a)lists.lavasoftware.org> Subject: [Lava-users] Why make the decision to limit job context in lava master? Message-ID: <DBBPR04MB63291E1DC4F5F202DEDFE16699060(a)DBBPR04MB6329.eurprd04.prod.outlook.com> Content-Type: text/plain; charset="utf-8" For 2019.04 version, we see next: Job context ----------- The schema validator is now checking the content of the `context` dictionary. Only the following keys are now allowed: * `arch`, `boot_console`, `boot_root`, `cpu`, `extra_options`, `guestfs_driveid`, `guestfs_interface`, `guestfs_size`, `machine`, `memory`, `model`, `monitor`, `netdevice`, `serial`, `vga` * `bootloader_prompt`, `console_device`, `extra_kernel_args`, `extra_nfsroot_args`, `kernel_loglevel`, `kernel_start_message`, `lava_test_results_dir`, `menu_interrupt_prompt`, `mustang_menu_list`, `test_character_delay`, `tftp_mac_address` Jobs using keys that are not listed in this list will be rejected. We usually set an customized context in job, and in device-type jinja2, use this context to just different value to set proper parameters. After this limit, all things break! So, my question is: lava could be designed to as a framework to give freedom to users to do their things as in the past, why we now enhance so many limits to users? And additional, and workaround for my scenario? Regards, Larry

5 years, 7 months

Problems with email notifications

by Frank, Matthias

Hi lava-users, my Debian machine running lava-server (2019.04) sends mails via msmtp/msmtp-mta. Mails from cron or apt work fine. But I got no mails from lava. Even when I submit a test job with a notify statement. notify: criteria: status: finished recipients: - to: method: email user: default verbosity: verbose The user default uses my real mail address not a system mail address. This is the Django log: INFO 2019-05-10 07:18:15,546 notifications [1343] sending email notification to XXXXXXX(a)YYYYYY.com ERROR 2019-05-10 07:18:15,557 notifications [Errno 99] Cannot assign requested address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/lava_scheduler_app/notifications.py", line 303, in send_notifications title, body, settings.SERVER_EMAIL, [recipient.email_address] File "/usr/lib/python3/dist-packages/django/core/mail/__init__.py", line 62, in send_mail return mail.send() File "/usr/lib/python3/dist-packages/django/core/mail/message.py", line 348, in send return self.get_connection(fail_silently).send_messages([self]) File "/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line 104, in send_messages new_conn_created = self.open() File "/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line 64, in open self.connection = self.connection_class(self.host, self.port, **connection_params) File "/usr/lib/python3.5/smtplib.py", line 251, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib/python3.5/smtplib.py", line 335, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib/python3.5/smtplib.py", line 306, in _get_socket self.source_address) File "/usr/lib/python3.5/socket.py", line 712, in create_connection raise err File "/usr/lib/python3.5/socket.py", line 703, in create_connection sock.connect(sa) OSError: [Errno 99] Cannot assign requested address WARNING 2019-05-10 07:18:15,558 notifications [1343] failed to send email notification to XXXXXXX(a)YYYYYY.com Lava has a problem with smtp. Is it necessary to have a complete smtp server on the local machine? Or is there a special smtp configuration? Greetings, Matthias

5 years, 7 months

Why make the decision to limit job context in lava master?

by Larry Shen

For 2019.04 version, we see next: Job context ----------- The schema validator is now checking the content of the `context` dictionary. Only the following keys are now allowed: * `arch`, `boot_console`, `boot_root`, `cpu`, `extra_options`, `guestfs_driveid`, `guestfs_interface`, `guestfs_size`, `machine`, `memory`, `model`, `monitor`, `netdevice`, `serial`, `vga` * `bootloader_prompt`, `console_device`, `extra_kernel_args`, `extra_nfsroot_args`, `kernel_loglevel`, `kernel_start_message`, `lava_test_results_dir`, `menu_interrupt_prompt`, `mustang_menu_list`, `test_character_delay`, `tftp_mac_address` Jobs using keys that are not listed in this list will be rejected. We usually set an customized context in job, and in device-type jinja2, use this context to just different value to set proper parameters. After this limit, all things break! So, my question is: lava could be designed to as a framework to give freedom to users to do their things as in the past, why we now enhance so many limits to users? And additional, and workaround for my scenario? Regards, Larry

5 years, 7 months

Board use rate in Lava?

by Denis HUMEAU

Dear users, I'd like to add metrics to scale my test farm. Basically I would like to check the occupation rate of the active boards in my farm, to be able to decide whether I need to add extra ones or not. Do you know if there is a simple way to do it? Best regards, Denis

5 years, 7 months

lxc test error

by jack lu

Hi, In my lava environment (debian 9.5 stretch, lava version:2019.04+stretch).When run a simple lxc test job.It happend a error: <http://192.168.101.205/scheduler/job/8#L83>Lxc container rootfs not found .. <http://192.168.101.205/scheduler/job/8#L101>LAVABug: This is probably a bug in LAVA, please report it. job definition is a simple lxc job from lava document: device_type: lxcjob_name: lxc-fedoratimeouts: job: minutes: 30 action: minutes: 5priority: mediumvisibility: public metadata: source: https://git.linaro.org/lava-team/refactoring.git path: lxc-fedora.yaml protocols: lava-lxc: name: pipeline-lxc-test distribution: fedora release: '26' arch: amd64 actions:- deploy: timeout: minutes: 30 to: lxc - boot: prompts: - '[root(a)(.*) /]#' timeout: minutes: 5 method: lxc - test: timeout: minutes: 5 definitions: - repository: http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests

5 years, 7 months

about multinode api parameter file overwritten

by jack lu

The multinode function of "pass data between devices".Note: "The message data is stored in a cache file which will be overwritten when the next synchronisation call is made. Ensure that your scripts make use of (or copy aside) any MultiNode cache data before calling any other MultiNode API helpers that may clear the cache." Does it mean follow example: roler1: .. steps: - lava-send ipv4 ip=192 - lava-send ipv4 ip=193 .. roler2: step2: - lava-wait ipv4 - ipdata=$(cat /tmp/lava_multi_node_cache.txt | cut -d = -f 2) .. After run, is content of roler2 ipdata 193?

5 years, 7 months

← Newer
1
...
40
41
42
43
44
45
46
...
92
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users