Hello to the Lava users,
I am the new Lava user. My aim is to use Lava (for now V1) setup for my testing, from the beginning just to hook-up my Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
As I am reading Lava framework, I got the following impression about the test suite: [1] I need to prepare BBB's U-Boot for the Lava U-Boot jinja2 setup; [2] I need to hook-up my EG-PMS2-LAN (energenie) to the Lava (to PO and POFF HW platform); [3] I need to hook-up ser2net interface (which I already have working over TCP) to the Lava. so Lava can control it; [4] I need to hook-up uImage, .dtb and ramdisk images to the Lava (which will be FTPed and set in memory for board setup and testing).
Question 1: Does manual have some Beagle Bone Black U-Boot default scripts, which should be provisioned to the BBB U-Boot for the correct Lava U-Boot behavior? Question 2: How do I do [1], [2], [3] and [4], does Lava manual have some explanation as working example how to do these points? Question 3: Anything else I missed for the proper Lava test setting?
Thank you in advance, Zoran
On 9 February 2018 at 09:00, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello to the Lava users,
I am the new Lava user. My aim is to use Lava (for now V1)
Welcome. Please take care with terminology. V1 is already dead, please don't start there. We will be unable to help you with your V1 setup. Your later comments refer to elements from V2, so please take care with which bits of documentation you are following.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
setup for my testing, from the beginning just to hook-up my Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
As I am reading Lava framework, I got the following impression about the test suite: [1] I need to prepare BBB's U-Boot for the Lava U-Boot jinja2 setup;
Jinja is V2 - so things are already getting confused. You can use the U-Boot that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
[2] I need to hook-up my EG-PMS2-LAN (energenie) to the Lava (to PO and POFF HW platform);
That looks like you have to manage the power control using a graphical interface and that's not going to work. You'll need to investigate how to control that unit using the command line only. Does it have a telnet API or other serial API? That will all be down to you to configure. You might want to look at using an APC PDU instead as those already have support in lavapdu (or can use SNMP if you configure the worker appropriately).
[3] I need to hook-up ser2net interface (which I already have working over TCP) to the Lava. so Lava can control it;
That goes into the device dictionary, as per the docs. Do you have the FTDI cable to attach to the BBB to get the serial connection? That will show up in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
[4] I need to hook-up uImage, .dtb and ramdisk images to the Lava (which will be FTPed and set in memory for board setup and testing).
TFTP, not FTP.
That is all managed by LAVA in the test job submission. TFTP is already configured.
Question 1: Does manual have some Beagle Bone Black U-Boot default scripts, which should be provisioned to the BBB U-Boot for the correct Lava U-Boot behavior?
Typically, for the BBB, we use the mainline U-Boot that comes from Debian. https://packages.debian.org/stretch/u-boot-omap
Question 2: How do I do [1], [2], [3] and [4], does Lava manual have some explanation as working example how to do these points?
Those are mostly specific to your local setup.
Question 3: Anything else I missed for the proper Lava test setting?
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Thank you in advance, Zoran
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Hello Neil,
Please, hold your horses. I am very new to all this, and I need some time to get to the Lava architecture, meaning to get in proper ways. In the sense, I'll try to rephrase the questions, and the working context, since your answers make me more confused than bring the viable solutions... :-(
After reading your email, there are the major addendums to this context, so let me rephrase/rework my initial email. _______
My aim is to use Lava V2 (since Lava V1 is not supported anymore). Let it be.
As I mentioned, I would like to use Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
But also, for the starters (seems the step in between) I can do QEMU (the problem is I have no idea how to do this outside of YOCTO Project).
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
I am running Lava v2017/7. Which supports ONLY V1. On Virtual Machine Debian Stretch (using VBox as VMM).
root@stretch:/home/vagrant# uname -a Linux stretch 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03) x86_64 GNU/Linux root@stretch:/home/vagrant# dpkg-query -W -f '${version}\n' 'lava-server' 2017.07
My problem here is that I can build the newest version of the Hashicorp debian/stretch64 using vagrant: https://app.vagrantup.com/debian/boxes/stretch64
But I am NOT able to include the newest Lava into this setup, since apt-get install Lava (and components) is bringing me Lava V1 (even very old version 2016.12-2)???
Q1: how I can bring here the newest Lava 2018.01 (only Version 2 compliant)? What the apt-get install Lava-2018.01 or similar command (I am Fedora monkey, as considering my host setup. Lava I am bringing/installing into the VM over VBox VMM)?
In other words, what is the specific command I need to use in scripts to bring proper Lava 2018.01??? Or any another command? Here is what is now used: sudo DEBIAN_FRONTEND=noninteractive apt-get -y install lava -t stretch-backports
Jinja is V2 - so things are already getting confused. You can use the U-Boot that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
I already use the newest 2017.11 BBB U-Boot, so anyway I need to tap (and change some strings) into the following files (as of my best understanding): root@stretch:/# find . -name base-uboot.jinja2 ./etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2 root@stretch:/# find . -name beaglebone-black.jinja2 ./etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 root@stretch:/# find . -name qemu.jinja2 ./etc/lava-server/dispatcher-config/device-types/qemu.jinja2 root@stretch:/#
Q2: I need here some examples, if they exist. How I can build these scripts, or should I use existing? Or to do something more to hook Lava to ser2net?
I already know: interrupt_prompt: {{ interrupt_prompt|default('Hit any key to stop autoboot') }} <<===== String MUST be changed for U-Boot 2017.11!
Q3: I see that base-uboot.jinja2 is a base file. Should I include it it other .jinja2 files using: {% extends 'base-uboot.jinja2' %}
That looks like you have to manage the power control using a graphical interface and that's not going to work...
Let us skip for now this question (I would like to simplify). PDU is for now not of importance. Focusing to make minimalistic approach to work.
That will show up in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
The whole ttyUSB0 using ser2net TCP is done already, works like a charm. In VM, as pass-through device.
Q4: do I need to use something special here to hook ser2net terminal to Lava?
root@stretch:/# cd /dev/serial root@stretch:/dev/serial# ls -al total 0 drwxr-xr-x 4 root root 80 Feb 9 11:29 . drwxr-xr-x 19 root root 3140 Feb 9 11:32 .. drwxr-xr-x 2 root root 60 Feb 9 11:29 by-id drwxr-xr-x 2 root root 60 Feb 9 11:29 by-path root@stretch:/dev/serial#
That is all managed by LAVA in the test job submission. TFTP is already configured.
in.tftp works like a charm (tftp-ing from VM to U-Boot BBB), as well as dhcpd (dnsmasq, also from VM). So, I have to trust you if you say that I do not need to do anything on that.
Typically, for the BBB, we use the mainline U-Boot that comes from Debian. https://packages.debian.org/stretch/u-boot-omap
In fact, this was not my question (I have latest 2017.11 U-Boot there on mmcblk1 which is /boot partition, works perfectly).
Q5: I wanted to know do I need to set up U-Boot scripts in some ways (as /etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 suggests, as example)??? In other words, make U-Boot environment more as .jinja2 suggests?
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Q6: So, what I need to do here? I use YOCTO project to build all BBB embedded Linux ingredients, uImage, .dtb and root tree, how I can use QEMU from YOCTO as independent .exe in Lava context??? Or, maybe, after all, I can try real image?!
I apologize for the long email. Some features are already solved (tftp, edhcp, ser2net), so we do not need to include them further (as I now understand).
Thank you, Zoran _______
On Fri, Feb 9, 2018 at 10:23 AM, Neil Williams neil.williams@linaro.org wrote:
On 9 February 2018 at 09:00, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello to the Lava users,
I am the new Lava user. My aim is to use Lava (for now V1)
Welcome. Please take care with terminology. V1 is already dead, please don't start there. We will be unable to help you with your V1 setup. Your later comments refer to elements from V2, so please take care with which bits of documentation you are following.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
setup for my testing, from the beginning just to hook-up my Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
As I am reading Lava framework, I got the following impression about the test suite: [1] I need to prepare BBB's U-Boot for the Lava U-Boot jinja2 setup;
Jinja is V2 - so things are already getting confused. You can use the U-Boot that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
[2] I need to hook-up my EG-PMS2-LAN (energenie) to the Lava (to PO and POFF HW platform);
That looks like you have to manage the power control using a graphical interface and that's not going to work. You'll need to investigate how to control that unit using the command line only. Does it have a telnet API or other serial API? That will all be down to you to configure. You might want to look at using an APC PDU instead as those already have support in lavapdu (or can use SNMP if you configure the worker appropriately).
[3] I need to hook-up ser2net interface (which I already have working over TCP) to the Lava. so Lava can control it;
That goes into the device dictionary, as per the docs. Do you have the FTDI cable to attach to the BBB to get the serial connection? That will show up in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
[4] I need to hook-up uImage, .dtb and ramdisk images to the Lava (which will be FTPed and set in memory for board setup and testing).
TFTP, not FTP.
That is all managed by LAVA in the test job submission. TFTP is already configured.
Question 1: Does manual have some Beagle Bone Black U-Boot default scripts, which should be provisioned to the BBB U-Boot for the correct Lava U-Boot behavior?
Typically, for the BBB, we use the mainline U-Boot that comes from Debian. https://packages.debian.org/stretch/u-boot-omap
Question 2: How do I do [1], [2], [3] and [4], does Lava manual have some explanation as working example how to do these points?
Those are mostly specific to your local setup.
Question 3: Anything else I missed for the proper Lava test setting?
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Thank you in advance, Zoran
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 9 February 2018 at 13:01, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello Neil,
Please, hold your horses. I am very new to all this, and I need some time to get to the Lava architecture, meaning to get in proper ways.
OK. We have tried to make the documentation as friendly as possible for this situation.
I reference the staging.validation.linaro.org docs here but you can adapt the URLs to localhost to read docs for your exact version of LAVA.
I've replied to your questions inline but it sounds as if you will find things easier to understand by ignoring the BBB work for now, follow the docs on setting up a QEMU device, submit the example first job from the documentation to run a x86_64 QEMU image inside LAVA and get that working. This will help with concepts like device-type templates and device dictionaries, device configuration and device creation. You can do all that, using LAVA V2, using the version of LAVA you already have installed.
0: The QEMU steps: https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi... https://staging.validation.linaro.org/static/docs/v2/pipeline-admin.html#cre... (Hint: as this is your local instance, just create the file in /etc/lava-server/dispatcher-config/devices/ as in the very last paragraph in that section.) https://staging.validation.linaro.org/static/docs/v2/first-job.html https://staging.validation.linaro.org/static/docs/v2/explain_first_job.html#...
Then come back to:
1: The upgrade steps, 2: configuring the PDU 3: configuring ser2net 4: the use of BBB using the example images, 5: finally move on to using your own images with the BBB.
In the sense, I'll try to rephrase the questions, and the working context, since your answers make me more confused than bring the viable solutions... :-(
After reading your email, there are the major addendums to this context, so let me rephrase/rework my initial email. _______
My aim is to use Lava V2 (since Lava V1 is not supported anymore). Let it be.
As I mentioned, I would like to use Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
But also, for the starters (seems the step in between) I can do QEMU (the problem is I have no idea how to do this outside of YOCTO Project).
There are example jobs in the LAVA documentation for QEMU. It is an ideal place to learn how a LAVA Test Shell works and what can be done. Having said that, your test shell definitions should aim to be portable. This means checking for the lava helpers in the $PATH and using them or using echo or logging etc if not. This allows you to create the scripts which run inside your test outside LAVA and use exactly the same scripts inside. All of this can be developed in QEMU.
We provide examples of the device dictionary, instructions on creating the device(s) for QEMU, example test jobs and permanent locations for the images to use in the test jobs on QEMU devices. All these images have been tested and continue to be tested regularly.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
I am running Lava v2017/7. Which supports ONLY V1. On Virtual Machine Debian Stretch (using VBox as VMM).
2017.7 supports V2. All releases after 2017.2 have usable V2 support.
V1 uses JSON test job submissions, V2 uses YAML test job submissions. Check the Help menu on your instance - the V2 documentation is available alongside V1.
Also check the documentation on LAVA repositories - those provide the way to upgrade stretch to the latest release, 2018.2
Don't confuse LAVA V2 with Python2 / Python3 support. LAVA V2 is also called "pipeline" and various parts of the UI refer to pipeline support.
root@stretch:/home/vagrant# uname -a Linux stretch 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03) x86_64 GNU/Linux root@stretch:/home/vagrant# dpkg-query -W -f '${version}\n' 'lava-server' 2017.07
I'm not sure how that was setup, the version available for stretch is in stretch-backports: https://packages.debian.org/source/stretch-backports/lava-dispatcher and https://packages.debian.org/source/stretch-backports/lava-server
2017.7-1~bpo9+1
My problem here is that I can build the newest version of the Hashicorp debian/stretch64 using vagrant: https://app.vagrantup.com/debian/boxes/stretch64
But I am NOT able to include the newest Lava into this setup, since apt-get install Lava (and components) is bringing me Lava V1 (even very old version 2016.12-2)???
https://staging.validation.linaro.org/static/docs/v2/installing_on_debian.ht...
Most of the LAVA instances are running the latest code on Stretch.
Q1: how I can bring here the newest Lava 2018.01 (only Version 2 compliant)? What the apt-get install Lava-2018.01 or similar command (I am Fedora monkey, as considering my host setup. Lava I am bringing/installing into the VM over VBox VMM)?
Create an apt source for:
deb https://images.validation.linaro.org/production-repo stretch-backports main
(Create a file in /etc/apt/sources.list.d/ called lava.list or similar with this lne - just make sure the filename ends in .list - then run apt update)
In other words, what is the specific command I need to use in scripts to bring proper Lava 2018.01??? Or any another command? Here is what is now used: sudo DEBIAN_FRONTEND=noninteractive apt-get -y install lava -t stretch-backports
stretch-backports has 2017.7-1~bpo9+1
You should follow the documentation and add the LAVA repositories if you want to get 2018.2
However, for your needs, 2017.7-1~bpo9+1 will be fine for your V2 development at this time and you can do the upgrade later.
Jinja is V2 - so things are already getting confused. You can use the
U-Boot
that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
I already use the newest 2017.11 BBB U-Boot, so anyway I need to tap (and change some strings) into the following files (as of my best understanding): root@stretch:/# find . -name base-uboot.jinja2 ./etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2 root@stretch:/# find . -name beaglebone-black.jinja2 ./etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 root@stretch:/# find . -name qemu.jinja2 ./etc/lava-server/dispatcher-config/device-types/qemu.jinja2 root@stretch:/#
The device-types don't need to be changed at this point. All you need to do is create the devices in the database and create a device dictionary - a set of commands which extend the device-type template into a configuration for a single device.
https://staging.validation.linaro.org/static/docs/v2/first-devices.html
https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi...
Q2: I need here some examples, if they exist. How I can build these scripts, or should I use existing? Or to do something more to hook Lava to ser2net?
https://staging.validation.linaro.org/static/docs/v2/first-devices.html#chec...
ser2net exposes a telnet port on a hostname. The connection_command is then set in the device dictionary. LAVA then calls telnet which opens the serial connection using the ser2net service.
I already know: interrupt_prompt: {{ interrupt_prompt|default('Hit any key to stop autoboot') }} <<===== String MUST be changed for U-Boot 2017.11!
That's an unusual build of U-Boot which changes the Hit any key interrupt prompt. If there is no prompt, you should set bootdelay to 10 so that you get the prompt to interrupt.
In your case, your device dictionary would contain (amongst other lines) a line like:
{% set interrupt_prompt = "Hit any key" %}
or whatever your interrupt prompt is.
Note: this isn't the same as the bootloader prompt (which could be U-Boot => or just => or similar) - that appears after the interrupt string.
This is a recent working test job on a BBB which shows the output: https://staging.validation.linaro.org/scheduler/job/209986 You can see the device dictionary here: https://git.linaro.org/lava/lava-lab.git/tree/staging.validation.linaro.org/... (that uses some features from 2018.2, it's just an example.)
Q3: I see that base-uboot.jinja2 is a base file. Should I include it it other .jinja2 files using: {% extends 'base-uboot.jinja2' %}
That is a step you do later when you add a new device-type.
For your device dictionary you need:
{% extends 'beaglebone-black.jinja2' %} {% set connection_command = 'telnet localhost 7105' %} {% set hard_reset_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command reboot --port 15' %} {% set power_off_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command off --port 15' %} {% set power_on_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command on --port 15' %}
(The last three lines are the PDU commands - yours will be quite different and you'll also need to change the port number according to how you've configured ser2net.)
That looks like you have to manage the power control using a graphical interface and that's not going to work...
Let us skip for now this question (I would like to simplify). PDU is for now not of importance. Focusing to make minimalistic approach to work.
PDU will required to get the BBB to work but not the QEMU. So stick to QEMU and the device-dictionary in the example to start with.
QEMU is much simpler as a device dictionary: {% extends 'qemu.jinja2' %} {% set mac_addr = '52:54:00:12:34:59' %} {% set memory = '1024' %}
That will show up in /dev/serial/by-id/ and that becomes part of the
ser2net configuration.
The whole ttyUSB0 using ser2net TCP is done already, works like a charm. In VM, as pass-through device.
Q4: do I need to use something special here to hook ser2net terminal to Lava?
root@stretch:/# cd /dev/serial root@stretch:/dev/serial# ls -al total 0 drwxr-xr-x 4 root root 80 Feb 9 11:29 . drwxr-xr-x 19 root root 3140 Feb 9 11:32 .. drwxr-xr-x 2 root root 60 Feb 9 11:29 by-id drwxr-xr-x 2 root root 60 Feb 9 11:29 by-path root@stretch:/dev/serial#
You need to configure ser2net with a line something like:
6007:telnet:0:/dev/serial/by-id/usb-FTDI_TTL232R-3V3_FTGNVQS6-if00-port0:115200 8DATABITS NONE 1STOPBIT banner LOCAL
That would then mean you run telnet localhost 6007 to connect to the ser2net service which provides access to that serial connection.
When you change the ser2net configuration (/etc/ser2net.conf) you need to tell the service to update: sudo service ser2net reload
That is all managed by LAVA in the test job submission. TFTP is already
configured.
in.tftp works like a charm (tftp-ing from VM to U-Boot BBB), as well as dhcpd (dnsmasq, also from VM). So, I have to trust you if you say that I do not need to do anything on that.
Typically, for the BBB, we use the mainline U-Boot that comes from
Debian.
In fact, this was not my question (I have latest 2017.11 U-Boot there on mmcblk1 which is /boot partition, works perfectly).
OK. The extra little hurdle with that version is that it requires changing the prompts from the defaults.
{% set interrupt_prompt = "Hit any key" %}
or whatever your interrupt prompt is.
Q5: I wanted to know do I need to set up U-Boot scripts in some ways (as /etc/lava-server/dispatcher-config/device-types/ beaglebone-black.jinja2 suggests, as example)??? In other words, make U-Boot environment more as .jinja2 suggests?
You set the overrides in the device dictionary for the device, not the device-type.
/etc/lava-server/dispatcher-config/devices/
Create the device dictionary there, using a filename which matches the name of the device in the database. e.g. bbb-01 would have a file bbb-01.jinja2
A lot of this will be easier to understand if you setup your first QEMU device by following the documentation step by step.
Start with QEMU, make sure that's working and get an understanding of
how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Q6: So, what I need to do here? I use YOCTO project to build all BBB embedded Linux ingredients, uImage, .dtb and root tree, how I can use QEMU from YOCTO as independent .exe in Lava context??? Or, maybe, after all, I can try real image?!
The reason BBB is recommended is because LAVA has example test jobs which use standard test images which are tested daily:
https://staging.validation.linaro.org/static/docs/v2/standard-test-jobs.html
https://staging.validation.linaro.org/static/docs/v2/standard-qemu-jessie.ht...
https://staging.validation.linaro.org/static/docs/v2/standard-armmp-ramdisk-...
https://staging.validation.linaro.org/static/docs/v2/examples/test-jobs/stan...
http://images.validation.linaro.org/snapshots.linaro.org/components/lava/sta...
Once you have those working, you can try your own images - i.e. change one thing at a time. Get things working with known working images so that you know that your PDU, ser2net and network infrastructure is working correctly before you add the extra step of your own images.
You wouldn't use a QEMU executable from Yocto, QEMU is installed by lava-dispatcher and will be used at that version. There are example image files to use with QEMU in LAVA, you don't need to build any yourself, at least initially. Start with x86_64 images to make things easier.
I apologize for the long email. Some features are already solved (tftp, edhcp, ser2net), so we do not need to include them further (as I now understand).
Thank you, Zoran _______
On Fri, Feb 9, 2018 at 10:23 AM, Neil Williams neil.williams@linaro.org wrote:
On 9 February 2018 at 09:00, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello to the Lava users,
I am the new Lava user. My aim is to use Lava (for now V1)
Welcome. Please take care with terminology. V1 is already dead, please
don't
start there. We will be unable to help you with your V1 setup. Your later comments refer to elements from V2, so please take care with which bits
of
documentation you are following.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
setup for my testing, from the beginning just to hook-up my Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper
context
for testing BBB HW.
As I am reading Lava framework, I got the following impression about the test suite: [1] I need to prepare BBB's U-Boot for the Lava U-Boot jinja2 setup;
Jinja is V2 - so things are already getting confused. You can use the
U-Boot
that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
[2] I need to hook-up my EG-PMS2-LAN (energenie) to the Lava (to PO and POFF HW platform);
That looks like you have to manage the power control using a graphical interface and that's not going to work. You'll need to investigate how to control that unit using the command line only. Does it have a telnet API
or
other serial API? That will all be down to you to configure. You might
want
to look at using an APC PDU instead as those already have support in
lavapdu
(or can use SNMP if you configure the worker appropriately).
[3] I need to hook-up ser2net interface (which I already have working
over
TCP) to the Lava. so Lava can control it;
That goes into the device dictionary, as per the docs. Do you have the
FTDI
cable to attach to the BBB to get the serial connection? That will show
up
in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
[4] I need to hook-up uImage, .dtb and ramdisk images to the Lava (which will be FTPed and set in memory for board setup and testing).
TFTP, not FTP.
That is all managed by LAVA in the test job submission. TFTP is already configured.
Question 1: Does manual have some Beagle Bone Black U-Boot default scripts, which should be provisioned to the BBB U-Boot for the correct
Lava
U-Boot behavior?
Typically, for the BBB, we use the mainline U-Boot that comes from
Debian.
https://packages.debian.org/stretch/u-boot-omap
Question 2: How do I do [1], [2], [3] and [4], does Lava manual have
some
explanation as working example how to do these points?
Those are mostly specific to your local setup.
Question 3: Anything else I missed for the proper Lava test setting?
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Thank you in advance, Zoran
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Hello Neil,
So let it be. You have convinced me to use QEMU, as the easiest example (?!) to this test suite, although I am not finding myself very well in it?!
Let me try to follow one step at the time.
You say:
0: The QEMU steps: https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi...
I do the following: lava-server manage device-types add qemu
It says that qemu device type is already added (python exception).
Then I do: lava-server manage device-types add qemu01
And it adds it somewhere (does not complain)???
So, I go to this link, and do the following. I go to: /etc/lava-server/dispatcher-config/device-types/
There I see the qemu.jinja2. So I have created the new device, called qemu01.jinja2 as:
{% extends 'qemu.jinja2' %} {% set mac_addr = '52:54:00:12:34:59' %} {% set memory = '1024' %}
So then, if I do: root@stretch:/etc/lava-server/dispatcher-config/device-types# lava-server manage device-types list -a
I see the device qemu01 there. Newly created. But I do NOT see qemu device.
It says: On the command line, you can add device types (for instance a QEMU type device with a hostname qemu01) using: lava-server manage devices add --device-type qemu --worker <worker> qemu01
What does mean name --worker <worker>?
Thank you, Zoran _______
On Fri, Feb 9, 2018 at 3:14 PM, Neil Williams neil.williams@linaro.org wrote:
On 9 February 2018 at 13:01, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello Neil,
Please, hold your horses. I am very new to all this, and I need some time to get to the Lava architecture, meaning to get in proper ways.
OK. We have tried to make the documentation as friendly as possible for this situation.
I reference the staging.validation.linaro.org docs here but you can adapt the URLs to localhost to read docs for your exact version of LAVA.
I've replied to your questions inline but it sounds as if you will find things easier to understand by ignoring the BBB work for now, follow the docs on setting up a QEMU device, submit the example first job from the documentation to run a x86_64 QEMU image inside LAVA and get that working. This will help with concepts like device-type templates and device dictionaries, device configuration and device creation. You can do all that, using LAVA V2, using the version of LAVA you already have installed.
0: The QEMU steps: https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi... https://staging.validation.linaro.org/static/docs/v2/pipeline-admin.html#cre... (Hint: as this is your local instance, just create the file in /etc/lava-server/dispatcher-config/devices/ as in the very last paragraph in that section.) https://staging.validation.linaro.org/static/docs/v2/first-job.html https://staging.validation.linaro.org/static/docs/v2/explain_first_job.html#...
Then come back to:
1: The upgrade steps, 2: configuring the PDU 3: configuring ser2net 4: the use of BBB using the example images, 5: finally move on to using your own images with the BBB.
In the sense, I'll try to rephrase the questions, and the working context, since your answers make me more confused than bring the viable solutions... :-(
After reading your email, there are the major addendums to this context, so let me rephrase/rework my initial email. _______
My aim is to use Lava V2 (since Lava V1 is not supported anymore). Let it be.
As I mentioned, I would like to use Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
But also, for the starters (seems the step in between) I can do QEMU (the problem is I have no idea how to do this outside of YOCTO Project).
There are example jobs in the LAVA documentation for QEMU. It is an ideal place to learn how a LAVA Test Shell works and what can be done. Having said that, your test shell definitions should aim to be portable. This means checking for the lava helpers in the $PATH and using them or using echo or logging etc if not. This allows you to create the scripts which run inside your test outside LAVA and use exactly the same scripts inside. All of this can be developed in QEMU.
We provide examples of the device dictionary, instructions on creating the device(s) for QEMU, example test jobs and permanent locations for the images to use in the test jobs on QEMU devices. All these images have been tested and continue to be tested regularly.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
I am running Lava v2017/7. Which supports ONLY V1. On Virtual Machine Debian Stretch (using VBox as VMM).
2017.7 supports V2. All releases after 2017.2 have usable V2 support.
V1 uses JSON test job submissions, V2 uses YAML test job submissions. Check the Help menu on your instance - the V2 documentation is available alongside V1.
Also check the documentation on LAVA repositories - those provide the way to upgrade stretch to the latest release, 2018.2
Don't confuse LAVA V2 with Python2 / Python3 support. LAVA V2 is also called "pipeline" and various parts of the UI refer to pipeline support.
root@stretch:/home/vagrant# uname -a Linux stretch 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3 (2017-12-03) x86_64 GNU/Linux root@stretch:/home/vagrant# dpkg-query -W -f '${version}\n' 'lava-server' 2017.07
I'm not sure how that was setup, the version available for stretch is in stretch-backports: https://packages.debian.org/source/stretch-backports/lava-dispatcher and https://packages.debian.org/source/stretch-backports/lava-server
2017.7-1~bpo9+1
My problem here is that I can build the newest version of the Hashicorp debian/stretch64 using vagrant: https://app.vagrantup.com/debian/boxes/stretch64
But I am NOT able to include the newest Lava into this setup, since apt-get install Lava (and components) is bringing me Lava V1 (even very old version 2016.12-2)???
https://staging.validation.linaro.org/static/docs/v2/installing_on_debian.ht...
Most of the LAVA instances are running the latest code on Stretch.
Q1: how I can bring here the newest Lava 2018.01 (only Version 2 compliant)? What the apt-get install Lava-2018.01 or similar command (I am Fedora monkey, as considering my host setup. Lava I am bringing/installing into the VM over VBox VMM)?
Create an apt source for:
deb https://images.validation.linaro.org/production-repo stretch-backports main
(Create a file in /etc/apt/sources.list.d/ called lava.list or similar with this lne - just make sure the filename ends in .list - then run apt update)
In other words, what is the specific command I need to use in scripts to bring proper Lava 2018.01??? Or any another command? Here is what is now used: sudo DEBIAN_FRONTEND=noninteractive apt-get -y install lava -t stretch-backports
stretch-backports has 2017.7-1~bpo9+1
You should follow the documentation and add the LAVA repositories if you want to get 2018.2
However, for your needs, 2017.7-1~bpo9+1 will be fine for your V2 development at this time and you can do the upgrade later.
Jinja is V2 - so things are already getting confused. You can use the U-Boot that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
I already use the newest 2017.11 BBB U-Boot, so anyway I need to tap (and change some strings) into the following files (as of my best understanding): root@stretch:/# find . -name base-uboot.jinja2 ./etc/lava-server/dispatcher-config/device-types/base-uboot.jinja2 root@stretch:/# find . -name beaglebone-black.jinja2 ./etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 root@stretch:/# find . -name qemu.jinja2 ./etc/lava-server/dispatcher-config/device-types/qemu.jinja2 root@stretch:/#
The device-types don't need to be changed at this point. All you need to do is create the devices in the database and create a device dictionary - a set of commands which extend the device-type template into a configuration for a single device.
https://staging.validation.linaro.org/static/docs/v2/first-devices.html
https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi...
Q2: I need here some examples, if they exist. How I can build these scripts, or should I use existing? Or to do something more to hook Lava to ser2net?
https://staging.validation.linaro.org/static/docs/v2/first-devices.html#chec...
ser2net exposes a telnet port on a hostname. The connection_command is then set in the device dictionary. LAVA then calls telnet which opens the serial connection using the ser2net service.
I already know: interrupt_prompt: {{ interrupt_prompt|default('Hit any key to stop autoboot') }} <<===== String MUST be changed for U-Boot 2017.11!
That's an unusual build of U-Boot which changes the Hit any key interrupt prompt. If there is no prompt, you should set bootdelay to 10 so that you get the prompt to interrupt.
In your case, your device dictionary would contain (amongst other lines) a line like:
{% set interrupt_prompt = "Hit any key" %}
or whatever your interrupt prompt is.
Note: this isn't the same as the bootloader prompt (which could be U-Boot => or just => or similar) - that appears after the interrupt string.
This is a recent working test job on a BBB which shows the output: https://staging.validation.linaro.org/scheduler/job/209986 You can see the device dictionary here: https://git.linaro.org/lava/lava-lab.git/tree/staging.validation.linaro.org/... (that uses some features from 2018.2, it's just an example.)
Q3: I see that base-uboot.jinja2 is a base file. Should I include it it other .jinja2 files using: {% extends 'base-uboot.jinja2' %}
That is a step you do later when you add a new device-type.
For your device dictionary you need:
{% extends 'beaglebone-black.jinja2' %} {% set connection_command = 'telnet localhost 7105' %} {% set hard_reset_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command reboot --port 15' %} {% set power_off_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command off --port 15' %} {% set power_on_command = '/usr/local/lab-scripts/snmp_pdu_control --hostname pdu15 --command on --port 15' %}
(The last three lines are the PDU commands - yours will be quite different and you'll also need to change the port number according to how you've configured ser2net.)
That looks like you have to manage the power control using a graphical interface and that's not going to work...
Let us skip for now this question (I would like to simplify). PDU is for now not of importance. Focusing to make minimalistic approach to work.
PDU will required to get the BBB to work but not the QEMU. So stick to QEMU and the device-dictionary in the example to start with.
QEMU is much simpler as a device dictionary: {% extends 'qemu.jinja2' %} {% set mac_addr = '52:54:00:12:34:59' %} {% set memory = '1024' %}
That will show up in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
The whole ttyUSB0 using ser2net TCP is done already, works like a charm. In VM, as pass-through device.
Q4: do I need to use something special here to hook ser2net terminal to Lava?
root@stretch:/# cd /dev/serial root@stretch:/dev/serial# ls -al total 0 drwxr-xr-x 4 root root 80 Feb 9 11:29 . drwxr-xr-x 19 root root 3140 Feb 9 11:32 .. drwxr-xr-x 2 root root 60 Feb 9 11:29 by-id drwxr-xr-x 2 root root 60 Feb 9 11:29 by-path root@stretch:/dev/serial#
You need to configure ser2net with a line something like:
6007:telnet:0:/dev/serial/by-id/usb-FTDI_TTL232R-3V3_FTGNVQS6-if00-port0:115200 8DATABITS NONE 1STOPBIT banner LOCAL
That would then mean you run telnet localhost 6007 to connect to the ser2net service which provides access to that serial connection.
When you change the ser2net configuration (/etc/ser2net.conf) you need to tell the service to update: sudo service ser2net reload
That is all managed by LAVA in the test job submission. TFTP is already configured.
in.tftp works like a charm (tftp-ing from VM to U-Boot BBB), as well as dhcpd (dnsmasq, also from VM). So, I have to trust you if you say that I do not need to do anything on that.
Typically, for the BBB, we use the mainline U-Boot that comes from Debian. https://packages.debian.org/stretch/u-boot-omap
In fact, this was not my question (I have latest 2017.11 U-Boot there on mmcblk1 which is /boot partition, works perfectly).
OK. The extra little hurdle with that version is that it requires changing the prompts from the defaults.
{% set interrupt_prompt = "Hit any key" %}
or whatever your interrupt prompt is.
Q5: I wanted to know do I need to set up U-Boot scripts in some ways (as /etc/lava-server/dispatcher-config/device-types/beaglebone-black.jinja2 suggests, as example)??? In other words, make U-Boot environment more as .jinja2 suggests?
You set the overrides in the device dictionary for the device, not the device-type.
/etc/lava-server/dispatcher-config/devices/
Create the device dictionary there, using a filename which matches the name of the device in the database. e.g. bbb-01 would have a file bbb-01.jinja2
A lot of this will be easier to understand if you setup your first QEMU device by following the documentation step by step.
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Q6: So, what I need to do here? I use YOCTO project to build all BBB embedded Linux ingredients, uImage, .dtb and root tree, how I can use QEMU from YOCTO as independent .exe in Lava context??? Or, maybe, after all, I can try real image?!
The reason BBB is recommended is because LAVA has example test jobs which use standard test images which are tested daily:
https://staging.validation.linaro.org/static/docs/v2/standard-test-jobs.html
https://staging.validation.linaro.org/static/docs/v2/standard-qemu-jessie.ht...
https://staging.validation.linaro.org/static/docs/v2/standard-armmp-ramdisk-...
https://staging.validation.linaro.org/static/docs/v2/examples/test-jobs/stan...
http://images.validation.linaro.org/snapshots.linaro.org/components/lava/sta...
Once you have those working, you can try your own images - i.e. change one thing at a time. Get things working with known working images so that you know that your PDU, ser2net and network infrastructure is working correctly before you add the extra step of your own images.
You wouldn't use a QEMU executable from Yocto, QEMU is installed by lava-dispatcher and will be used at that version. There are example image files to use with QEMU in LAVA, you don't need to build any yourself, at least initially. Start with x86_64 images to make things easier.
I apologize for the long email. Some features are already solved (tftp, edhcp, ser2net), so we do not need to include them further (as I now understand).
Thank you, Zoran _______
On Fri, Feb 9, 2018 at 10:23 AM, Neil Williams neil.williams@linaro.org wrote:
On 9 February 2018 at 09:00, Zoran S zoran.stojsavljevic.de@gmail.com wrote:
Hello to the Lava users,
I am the new Lava user. My aim is to use Lava (for now V1)
Welcome. Please take care with terminology. V1 is already dead, please don't start there. We will be unable to help you with your V1 setup. Your later comments refer to elements from V2, so please take care with which bits of documentation you are following.
What version of LAVA are you running? On Debian Jessie or Debian Stretch?
setup for my testing, from the beginning just to hook-up my Beagle Bone Black (BBB) to Lava worker, and from Lava apache to set the proper context for testing BBB HW.
As I am reading Lava framework, I got the following impression about the test suite: [1] I need to prepare BBB's U-Boot for the Lava U-Boot jinja2 setup;
Jinja is V2 - so things are already getting confused. You can use the U-Boot that comes with the BBB but you will need to account for any changes in prompts etc. in the device configuration.
[2] I need to hook-up my EG-PMS2-LAN (energenie) to the Lava (to PO and POFF HW platform);
That looks like you have to manage the power control using a graphical interface and that's not going to work. You'll need to investigate how to control that unit using the command line only. Does it have a telnet API or other serial API? That will all be down to you to configure. You might want to look at using an APC PDU instead as those already have support in lavapdu (or can use SNMP if you configure the worker appropriately).
[3] I need to hook-up ser2net interface (which I already have working over TCP) to the Lava. so Lava can control it;
That goes into the device dictionary, as per the docs. Do you have the FTDI cable to attach to the BBB to get the serial connection? That will show up in /dev/serial/by-id/ and that becomes part of the ser2net configuration.
[4] I need to hook-up uImage, .dtb and ramdisk images to the Lava (which will be FTPed and set in memory for board setup and testing).
TFTP, not FTP.
That is all managed by LAVA in the test job submission. TFTP is already configured.
Question 1: Does manual have some Beagle Bone Black U-Boot default scripts, which should be provisioned to the BBB U-Boot for the correct Lava U-Boot behavior?
Typically, for the BBB, we use the mainline U-Boot that comes from Debian. https://packages.debian.org/stretch/u-boot-omap
Question 2: How do I do [1], [2], [3] and [4], does Lava manual have some explanation as working example how to do these points?
Those are mostly specific to your local setup.
Question 3: Anything else I missed for the proper Lava test setting?
Start with QEMU, make sure that's working and get an understanding of how that works with the device dictionary, test job submission, test shell definitions and general LAVA UI usage.
Thank you in advance, Zoran
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Hello,
0: The QEMU steps:
https://staging.validation.linaro.org/static/docs/v2/ first-devices.html#adding-a-dictionary-to-the-first-qemu-device
I do the following: lava-server manage device-types add qemu
This is expected if the corresponding device-type (qemu) was already added. Which is the case in your instance.
It says that qemu device type is already added (python exception).
I sent a patch to print an error message instead of the python exception.
Then I do:
lava-server manage device-types add qemu01
You are mixing "device-types" and "devices". You should add a device here.
There I see the qemu.jinja2. So I have created the new device, called qemu01.jinja2 as:
{% extends 'qemu.jinja2' %} {% set mac_addr = '52:54:00:12:34:59' %} {% set memory = '1024' %}
Same here, this is a device configuration that should go into /etc/lava-server/dispatcher-config/devices/
So then, if I do:
root@stretch:/etc/lava-server/dispatcher-config/device-types# lava-server manage device-types list -a
I see the device qemu01 there. Newly created. But I do NOT see qemu device.
This command will only list the device-types not the devices. To list the devices, use
lava-server manage devices list
It says:
On the command line, you can add device types (for instance a QEMU type device with a hostname qemu01) using: lava-server manage devices add --device-type qemu --worker <worker> qemu01
The worker (or dispatcher) is the name of the machine that will run the qemu or (for boards) which is connected to the device you are creating.
You can see the workers that are actually active in your instance with lava-server manage workers list
Rgds
Let me go with small steps (so you do not loose too much time):
[1] I moved qemu01.jinja2 from device-types/ to device/
[2] Now I have there two devices (device types are on parenthesis): root@stretch:/etc/lava-server/dispatcher-config/devices# lava-server manage devices list Available devices: * bbb01 (beaglebone-black) Idle * qemu01 (qemu) Idle
[3] root@stretch:/etc/lava-server/dispatcher-config/devices# lava-server manage device-types list -a Installed device types: * beaglebone-black (1 devices) * qemu (1 devices) * qemu01 (0 devices) - No V2 template. <<=========================== how to remove/delete??? Available V2 device types: [snap]
Q1: how I can remove/delete qemu01 device-type I added by mistake?
[4] root@stretch:/etc/lava-server/dispatcher-config/devices# lava-server manage workers list Workers: * stretch.localdomain (2 devices) (master)
[5] Now, the following command: lava-server manage devices add --device-type qemu --worker stretch.localdomain qemu01 <<================
qemu01 here is "hostname". Could you, please, explain this? It seems to me that qemu01 (device type qemu) should be target, right???
Thank you, Zoran _______
On Mon, Feb 12, 2018 at 3:38 PM, Remi Duraffort remi.duraffort@linaro.org wrote:
Hello,
0: The QEMU steps:
https://staging.validation.linaro.org/static/docs/v2/first-devices.html#addi...
I do the following: lava-server manage device-types add qemu
This is expected if the corresponding device-type (qemu) was already added. Which is the case in your instance.
It says that qemu device type is already added (python exception).
I sent a patch to print an error message instead of the python exception.
Then I do: lava-server manage device-types add qemu01
You are mixing "device-types" and "devices". You should add a device here.
There I see the qemu.jinja2. So I have created the new device, called qemu01.jinja2 as:
{% extends 'qemu.jinja2' %} {% set mac_addr = '52:54:00:12:34:59' %} {% set memory = '1024' %}
Same here, this is a device configuration that should go into /etc/lava-server/dispatcher-config/devices/
So then, if I do: root@stretch:/etc/lava-server/dispatcher-config/device-types# lava-server manage device-types list -a
I see the device qemu01 there. Newly created. But I do NOT see qemu device.
This command will only list the device-types not the devices. To list the devices, use
lava-server manage devices list
It says: On the command line, you can add device types (for instance a QEMU type device with a hostname qemu01) using: lava-server manage devices add --device-type qemu --worker <worker> qemu01
The worker (or dispatcher) is the name of the machine that will run the qemu or (for boards) which is connected to the device you are creating.
You can see the workers that are actually active in your instance with lava-server manage workers list
Rgds
-- Rémi Duraffort
[1] I moved qemu01.jinja2 from device-types/ to device/
That's right has qemu01.jinja2 describe a device and not a device-type.
Q1: how I can remove/delete qemu01 device-type I added by mistake?
In the admin interface: http://localhost/admin/lava_scheduler_app/device/
[5] Now, the following command:
lava-server manage devices add --device-type qemu --worker stretch.localdomain qemu01 <<================
qemu01 here is "hostname". Could you, please, explain this? It seems to me that qemu01 (device type qemu) should be target, right???
The device "hostname" is just the name of the device.
OK. I did it from the Django Administration:
This is what I did: root@stretch:/etc/lava-server/dispatcher-config/devices# lava-server manage devices add --device-type qemu01 --worker stretch.localdomain host1 root@stretch:/etc/lava-server/dispatcher-config/devices#
[There I deleted from the localhost:8080 admin interface, first adding qemu01 type as nonexistent host1 device]
root@stretch:/etc/lava-server/dispatcher-config/devices# lava-server manage device-types list -a Installed device types: * beaglebone-black (1 devices) * qemu (1 devices) Available V2 device types: [snap]
Now, my question is the following: how I add * qemu (1 devices) to the test? Actually, I see: lava-server manage devices add --device-type qemu --worker stretch.localdomain qemu01 lava-server manage devices add --device-type beaglebone-black --worker stretch.localdomain bbb01
From: root@stretch:/etc/lava-server/dispatcher-config/devices# ls -al total 16 drwxr-xr-x 2 lavaserver lavaserver 4096 Jan 30 12:45 . drwxr-xr-x 5 lavaserver lavaserver 4096 Feb 12 15:43 .. -rw-r--r-- 1 lavaserver lavaserver 136 Jan 30 12:45 bbb01.jinja2 -rw-r--r-- 1 lavaserver lavaserver 119 Jan 30 12:45 qemu01.jinja2 root@stretch:/etc/lava-server/dispatcher-config/devices#
Am I correct here? _______
Actually, I already have two devices in the following localhost web location: http://localhost:8080/admin/lava_scheduler_app/device/
I would like to submit the following test for my qemu01 hiost of device-type qemu: https://staging.validation.linaro.org/static/docs/v2/first-job.html
Where I do this on the http://localhost:8080/admin ?
Thank you, Zoran _______
On Mon, Feb 12, 2018 at 4:18 PM, Remi Duraffort remi.duraffort@linaro.org wrote:
[1] I moved qemu01.jinja2 from device-types/ to device/
That's right has qemu01.jinja2 describe a device and not a device-type.
Q1: how I can remove/delete qemu01 device-type I added by mistake?
In the admin interface: http://localhost/admin/lava_scheduler_app/device/
[5] Now, the following command: lava-server manage devices add --device-type qemu --worker stretch.localdomain qemu01 <<================
qemu01 here is "hostname". Could you, please, explain this? It seems to me that qemu01 (device type qemu) should be target, right???
The device "hostname" is just the name of the device.
-- Rémi Duraffort
Actually, I already have two devices in the following localhost web location: http://localhost:8080/admin/lava_scheduler_app/device/
Also visible in the normal web interface: http://localhost:8080/scheduler/alldevices
I would like to submit the following test for my qemu01 hiost of
device-type qemu: https://staging.validation.linaro.org/static/docs/v2/first-job.html
Where I do this on the http://localhost:8080/admin ?
http://localhost:8080/scheduler/jobsubmit
Hello Remi,
Thank you for the help. It is for me a huge help in Lava understanding.
I went to the following link: https://staging.validation.linaro.org/static/docs/v2/first-job.html
BTW, where I can find such job in the local lava directory structures (so I can also use CLI commands)?
And drag and drop YAML file in: http://localhost:8080/scheduler/jobsubmit
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
So, do you have some another viable stretch image I can pick up and test? Or, perhaps, I can extract the image from YOCTO Project (I need to find out where in YOCTO Project they keep QEMU images after bitbake core-image-minimal, for x86_64 (host architecture), for example)?
The transcript of the failure follows.
Should I place QEMU stretch image somewhere locally, in local Lava directories (CLI question)? Where exactly?
Thank you, Zoran _______
lava-dispatcher, installed at version: 2017.7-1~bpo9+1start: 0 validateValidating that https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz existsRessource not availableqemu-system-x86, installed at version: 1:2.8+dfsg-6+deb9u3Invalid job definitionInvalid job data: ["Unable to get 'https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz': HTTPSConnectionPool(host='images.validation.linaro.org', port=443): Max retries exceeded with url: /kvm/standard/stretch-2.img.gz (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4db0e0dc50>: Failed to establish a new connection: [Errno 110] Connection timed out',))"] validate duration: 15.37case: validate definition: lava result: fail Cleaning after the jobRoot tmp directory removed at /var/lib/lava/dispatcher/tmp/1JobError: Your job cannot terminate cleanly.case: job definition: lava error_type: Job result: fail error_msg: Invalid job data: ["Unable to get 'https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz': HTTPSConnectionPool(host='images.validation.linaro.org', port=443): Max retries exceeded with url: /kvm/standard/stretch-2.img.gz (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f4db0e0dc50>: Failed to establish a new connection: [Errno 110] Connection timed out',))"] _______
On Tue, Feb 13, 2018 at 8:30 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
Actually, I already have two devices in the following localhost web location: http://localhost:8080/admin/lava_scheduler_app/device/
Also visible in the normal web interface: http://localhost:8080/scheduler/alldevices
I would like to submit the following test for my qemu01 hiost of device-type qemu: https://staging.validation.linaro.org/static/docs/v2/first-job.html
Where I do this on the http://localhost:8080/admin ?
http://localhost:8080/scheduler/jobsubmit
-- Rémi Duraffort LAVA Team
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
The dispatcher can connect to internet. because this resource is valid (you can check in your browser).
Hello Remi,
The proxy settings are the problem. Do you have anywhere in Lava server explicit place where you set the proxies?
Best Regards, Zoran _______
On Wed, Feb 14, 2018 at 10:14 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
The dispatcher can connect to internet. because this resource is valid (you can check in your browser).
-- Rémi Duraffort LAVA Team
In /etc/lava-server/env.yaml which describes the environment that the dispatcher will use to run the jobs.
2018-02-14 11:13 GMT+01:00 Zoran S zoran.stojsavljevic.de@gmail.com:
Hello Remi,
The proxy settings are the problem. Do you have anywhere in Lava server explicit place where you set the proxies?
Best Regards, Zoran _______
On Wed, Feb 14, 2018 at 10:14 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
The dispatcher can connect to internet. because this resource is valid
(you
can check in your browser).
-- Rémi Duraffort LAVA Team
This does not work.
According what I read from the /etc/lava-server/env.yaml, I did the following to the QEMU first job.
It again failed in the same place.
Could you, please, advise???
Thank you, Zoran _______
# Your first LAVA JOB definition for an x86_64 QEMU device_type: qemu job_name: QEMU pipeline, first job
timeouts: job: minutes: 15 action: minutes: 5 connection: minutes: 2 priority: medium visibility: public
# context allows specific values to be overridden or included context: # tell the qemu template which architecture is being tested # the template uses that to ensure that qemu-system-x86_64 is executed. arch: amd64 http_proxy: http://xxx.xxx.xxx.xxx:xxx https_proxy: https://xxx.xxx.xxx.xxx:xxx ftp_proxy: http://xxx.xxx.xxx.xxx:xxx
metadata: # please change these fields when modifying this job for your own tests. docs-source: first-job docs-filename: qemu-pipeline-first-job.yaml
# ACTION_BLOCK actions: - deploy: timeout: minutes: 5h to: tmpfs images: rootfs: image_arg: -drive format=raw,file={rootfs} url: https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz compression: gz http_proxy: http://xxx.xxx.xxx.xxx:xxx https_proxy: https://xxx.xxx.xxx.xxx:xxx ftp_proxy: http://xxx.xxx.xxx.xxx:xxx os: debian
# BOOT_BLOCK - boot: timeout: minutes: 2 method: qemu media: tmpfs prompts: ["root@debian:"] auto_login: login_prompt: "login:" username: root
- test: timeout: minutes: 5 definitions: - repository: http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests - repository: https://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/single-node/singlenode03.yaml name: singlenode-advanced
On Wed, Feb 14, 2018 at 11:36 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
In /etc/lava-server/env.yaml which describes the environment that the dispatcher will use to run the jobs.
2018-02-14 11:13 GMT+01:00 Zoran S zoran.stojsavljevic.de@gmail.com:
Hello Remi,
The proxy settings are the problem. Do you have anywhere in Lava server explicit place where you set the proxies?
Best Regards, Zoran _______
On Wed, Feb 14, 2018 at 10:14 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
The dispatcher can connect to internet. because this resource is valid (you can check in your browser).
-- Rémi Duraffort LAVA Team
-- Rémi Duraffort LAVA Team
YES, yes... I see how this should be solved.
It could be solved also directly using urllib3 python code, going directly to Python interpreter (I have tried it, it does work).
Here: https://stackoverflow.com/questions/31151615/how-to-handle-proxies-in-urllib...
In other words, Lava does NOT read VM's ENV variables. It ignores them. Python reads /etc/lava-server/env.yaml as setup proxy file!
Zoran _______
On Wed, Feb 14, 2018 at 11:36 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
In /etc/lava-server/env.yaml which describes the environment that the dispatcher will use to run the jobs.
2018-02-14 11:13 GMT+01:00 Zoran S zoran.stojsavljevic.de@gmail.com:
Hello Remi,
The proxy settings are the problem. Do you have anywhere in Lava server explicit place where you set the proxies?
Best Regards, Zoran _______
On Wed, Feb 14, 2018 at 10:14 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
The job failed immediately. Since the following message poped up in my localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz -> Resources not available!
The dispatcher can connect to internet. because this resource is valid (you can check in your browser).
-- Rémi Duraffort LAVA Team
-- Rémi Duraffort LAVA Team
Not exactly.
1/ The process that is deploying, booting and testing a DUT (Device Under test) is called lava-run. lava-run environment variables are controlled by /etc/lava-server/env.yaml
By default, all environment variables are removed and only a small set of variables are added. This helps to make executions reproducible between dispatchers and instances.
So if you need an environment variable to be set, then you have to add it to /etc/lava-server/env.yaml
2/ On the DUT itself, by default, we don't add or change any environment variable. Because that's the user responsibility. However, if /etc/lava-server/env.dut.yaml does exists, it will be used to add environment variables to the DUT shell. To create the list of environment variables to add to the DUT, we take the full environment from lava-run (as defined by /etc/lava-server/env.yaml) and we apply the rules from env.dut.yaml.
I hope that does help you to understand how environment is setup in lava.
Rgds
2018-02-14 12:53 GMT+01:00 Zoran S zoran.stojsavljevic.de@gmail.com:
YES, yes... I see how this should be solved.
It could be solved also directly using urllib3 python code, going directly to Python interpreter (I have tried it, it does work).
Here: https://stackoverflow.com/questions/31151615/how-to- handle-proxies-in-urllib3
In other words, Lava does NOT read VM's ENV variables. It ignores them. Python reads /etc/lava-server/env.yaml as setup proxy file!
Zoran _______
On Wed, Feb 14, 2018 at 11:36 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
In /etc/lava-server/env.yaml which describes the environment that the dispatcher will use to run the jobs.
2018-02-14 11:13 GMT+01:00 Zoran S zoran.stojsavljevic.de@gmail.com:
Hello Remi,
The proxy settings are the problem. Do you have anywhere in Lava server explicit place where you set the proxies?
Best Regards, Zoran _______
On Wed, Feb 14, 2018 at 10:14 AM, Remi Duraffort remi.duraffort@linaro.org wrote:
The job failed immediately. Since the following message poped up in
my
localhost browser:
https://images.validation.linaro.org/kvm/standard/stretch-2.img.gz
->
Resources not available!
The dispatcher can connect to internet. because this resource is valid (you can check in your browser).
-- Rémi Duraffort LAVA Team
-- Rémi Duraffort LAVA Team
lava-users@lists.lavasoftware.org
-
Neil Williams -
Remi Duraffort -
Zoran S