Hello everyone,
in our current project we have some devices that are not "directly" supported by lava. I would like to ask for you opinion on which would be the correct way to proceed.
The main problem is that our devices have a EFI implementation in the firmware that is making the task of installing uboot very hard. To avoid this, we thought about serving the image through an "emulated" usb stick. On that regard, we made some progress by setting up a secondary device that would use the linux g_mass_storage module to serve the image through the USB otg port.
Our setup is then:
1) The actual testing device 2) The secondary device which only serves the image with g_mass_storage 3) The host machine running the lava-slave application.
We thought that we could add a new device-type template to the lava server that would somehow override the deployment and boot actions to address our setup. We tried to look into the base.jinja2 file for some sort of entry point that would allow us to, for example, run a script that would first send the image to the secondary device and secondly run the g_mass_storage module with the image file.
Unfortunately, it is still not clear for us how exactly we could inject/override those steps and ignore for instance the boot loader commands.
The first question would be, are we going in the right direction? Is there any piece of documentation that would describe a similar problem?
What would be your recommendations on this topic?
thanks a lot in advance.
Best, Alfonso
Hi Alfonso,
On Wed, Nov 08, 2017 at 04:24:41PM +0000, Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) wrote:
in our current project we have some devices that are not "directly" supported by lava. I would like to ask for you opinion on which would be the correct way to proceed.
The main problem is that our devices have a EFI implementation in the firmware that is making the task of installing uboot very hard. To avoid this, we thought about serving the image through an "emulated" usb stick. On that regard, we made some progress by setting up a secondary device that would use the linux g_mass_storage module to serve the image through the USB otg port.
Our setup is then:
- The actual testing device
- The secondary device which only serves the image with g_mass_storage
- The host machine running the lava-slave application.
We thought that we could add a new device-type template to the lava server that would somehow override the deployment and boot actions to address our setup. We tried to look into the base.jinja2 file for some sort of entry point that would allow us to, for example, run a script that would first send the image to the secondary device and secondly run the g_mass_storage module with the image file.
It sounds like you're trying to do something like what's supported via LAVA's "secondary media" support?
https://validation.linaro.org/static/docs/v2/secondary-media.html
although you might be adding more complexity to your tests that way. If your devices already boot via UEFI, could you configure them to net-boot from UEFI instead, and run your tests that way?
Cheers,
Hi Steve,
thanks for the reply. The documentation says that it is possible to deploy the image to a media external to the device, but it is still not clear to me how exactly that works.
Is that referring to a shared device between the lava-slave host and the test device?
Thanks again.
Best, Alfonso
In such case, I imagine that the bootloader is the one responsible to receive and store the image. Is that right?
Is it possible to use linux as the bootloader? does lava consider that case?
In that case i would only need one device with some external media attached to it running linux.
On 08.11.2017 17:38, Steve McIntyre wrote:
Hi Alfonso,
On Wed, Nov 08, 2017 at 04:24:41PM +0000, Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) wrote:
in our current project we have some devices that are not "directly" supported by lava. I would like to ask for you opinion on which would be the correct way to proceed.
The main problem is that our devices have a EFI implementation in the firmware that is making the task of installing uboot very hard. To avoid this, we thought about serving the image through an "emulated" usb stick. On that regard, we made some progress by setting up a secondary device that would use the linux g_mass_storage module to serve the image through the USB otg port.
Our setup is then:
- The actual testing device
- The secondary device which only serves the image with g_mass_storage
- The host machine running the lava-slave application.
We thought that we could add a new device-type template to the lava server that would somehow override the deployment and boot actions to address our setup. We tried to look into the base.jinja2 file for some sort of entry point that would allow us to, for example, run a script that would first send the image to the secondary device and secondly run the g_mass_storage module with the image file.
It sounds like you're trying to do something like what's supported via LAVA's "secondary media" support?
https://validation.linaro.org/static/docs/v2/secondary-media.html
although you might be adding more complexity to your tests that way. If your devices already boot via UEFI, could you configure them to net-boot from UEFI instead, and run your tests that way?
Cheers,
On 9 November 2017 at 16:05, Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) alfonso.ros-dos-santos@evosoft.com wrote:
Hi Steve,
thanks for the reply. The documentation says that it is possible to deploy the image to a media external to the device, but it is still not clear to me how exactly that works.
Not so much external as just not the primary boot media. (Primary as in "where the bootloader is installed")
Is that referring to a shared device between the lava-slave host and the test device?
Secondary media relates to things like a SATA drive on a board which has the bootloader (U-Boot or UEFI) on flash storage elsewhere. The second media is still physically local to the device, it just isn't the top of the boot order. What matters is that the bootloader is able to read files from that media and boot the device using those files.
Thanks again.
Best, Alfonso
In such case, I imagine that the bootloader is the one responsible to receive and store the image. Is that right?
Most bootloaders aren't capable of writing large images to media like SATA or even USB sticks. Instead, the idea of secondary media is to use the bootloader to boot Linux with a ramdisk or NFS or similar. That environment provides enough tools to mount and write to the secondary media. Then, the device is rebooted and the bootloader reads the new kernel and supporting files from the secondary media instead of the default boot.
Is it possible to use linux as the bootloader? does lava consider that case?
It is possible but we don't have explicit support for interacting with Linux directly. Using Linux to drop to an initramfs prompt is certainly supported.
In that case i would only need one device with some external media attached to it running linux.
On 08.11.2017 17:38, Steve McIntyre wrote:
Hi Alfonso,
On Wed, Nov 08, 2017 at 04:24:41PM +0000, Ros Dos Santos, Alfonso (CT
RDA DS EVO OPS DIA SE 1) wrote:
in our current project we have some devices that are not "directly" supported by lava. I would like to ask for you opinion on which would be the correct way to proceed.
The main problem is that our devices have a EFI implementation in the firmware that is making the task of installing uboot very hard.
LAVA has UEFI support - this can be downloading Grub using PXE or it can be executing something like Grub or fastboot as an EFI application. Then that bootloader does the work of deploying and booting the test files - this way you might not need to think about secondary media. You can simply netboot the device and chain into a different bootloader which doesn't have the restrictions living inside the firmware.
To avoid
this, we thought about serving the image through an "emulated" usb stick. On that regard, we made some progress by setting up a secondary device that would use the linux g_mass_storage module to serve the image through the USB otg port.
Our setup is then:
- The actual testing device
- The secondary device which only serves the image with g_mass_storage
- The host machine running the lava-slave application.
We thought that we could add a new device-type template to the lava server that would somehow override the deployment and boot actions to address our setup.
Please read some of the developer documentation. This isn't about overriding deployment or boot actions, this is about arranging a new list of actions in a dedicated pipeline for this board. The list would including a few new action classes written for this method.
We tried to look into the base.jinja2 file for some
sort of entry point that would allow us to, for example, run a script that would first send the image to the secondary device and secondly run the g_mass_storage module with the image file.
base.jinja2 isn't going to be enough for this. The jinja2 templates are already handled as code and this kind of work is going to need other code changes inside lava-dispatcher as well.
It sounds like you're trying to do something like what's supported via LAVA's "secondary media" support?
https://validation.linaro.org/static/docs/v2/secondary-media.html
although you might be adding more complexity to your tests that way. If your devices already boot via UEFI, could you configure them to net-boot from UEFI instead, and run your tests that way?
Netbooting from UEFI is a much simpler way of doing things. Secondary media is complex. It requires two boot operations and two deployment operations per test job - sometimes writing a rootfs to secondary media can be a lengthy process. A USB stick is not generally known for fast writes.
Cheers,
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Hi Neil,
thank you very much for the information. It was of great use. We have solved our issue just as you indicated, by creating a custom deployment strategy for our specific device. In this deployment strategy we simply run a script that will move the image and mount it into the device with the usb otg port using the USB Gadget module.
Here is the strategy:
79 class USBGadgetDeploymentAction(DeployAction): 80 81 def __init__(self): 82 super(USBGadgetDeploymentAction, self).__init__() 83 self.name = 'usb-gadget-deploy' 84 self.description = "deploy images using the fake usb device" 85 self.summary = "deploy images" 86 87 def populate(self, parameters): 88 self.internal_pipeline = Pipeline(parent=self, job=self.job, parameters=parameters) 89 path = self.mkdtemp() 90 91 # Download the image 92 self.internal_pipeline.add_action(DownloaderAction('image', path)) 93 94 if self.test_needs_overlay(parameters): 95 self.internal_pipeline.add_action(OverlayAction()) 96 self.internal_pipeline.add_action(ApplyOverlayImage()) 97 98 host = parameters['image']['usb_gadget_host'] 99 self.internal_pipeline.add_action(USBGadgetScriptAction(host))
We just introduced this "usb_gadget_host" parameter to the image parameters which is the device where to mount the image. And finally our USBGadgetScriptAction finishes the job. The boot and test strategies can be used quite independently. For the boot we use a minimal strategy to simply restart the device.
Thanks again.
Best,
Alfonso
On 09.11.2017 18:10, Neil Williams wrote: On 9 November 2017 at 16:05, Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) <alfonso.ros-dos-santos@evosoft.commailto:alfonso.ros-dos-santos@evosoft.com> wrote: Hi Steve,
thanks for the reply. The documentation says that it is possible to deploy the image to a media external to the device, but it is still not clear to me how exactly that works.
Not so much external as just not the primary boot media. (Primary as in "where the bootloader is installed")
Is that referring to a shared device between the lava-slave host and the test device?
Secondary media relates to things like a SATA drive on a board which has the bootloader (U-Boot or UEFI) on flash storage elsewhere. The second media is still physically local to the device, it just isn't the top of the boot order. What matters is that the bootloader is able to read files from that media and boot the device using those files.
Thanks again.
Best, Alfonso
In such case, I imagine that the bootloader is the one responsible to receive and store the image. Is that right?
Most bootloaders aren't capable of writing large images to media like SATA or even USB sticks. Instead, the idea of secondary media is to use the bootloader to boot Linux with a ramdisk or NFS or similar. That environment provides enough tools to mount and write to the secondary media. Then, the device is rebooted and the bootloader reads the new kernel and supporting files from the secondary media instead of the default boot.
Is it possible to use linux as the bootloader? does lava consider that case?
It is possible but we don't have explicit support for interacting with Linux directly. Using Linux to drop to an initramfs prompt is certainly supported.
In that case i would only need one device with some external media attached to it running linux.
On 08.11.2017 17:38, Steve McIntyre wrote:
Hi Alfonso,
On Wed, Nov 08, 2017 at 04:24:41PM +0000, Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) wrote:
in our current project we have some devices that are not "directly" supported by lava. I would like to ask for you opinion on which would be the correct way to proceed.
The main problem is that our devices have a EFI implementation in the firmware that is making the task of installing uboot very hard.
LAVA has UEFI support - this can be downloading Grub using PXE or it can be executing something like Grub or fastboot as an EFI application. Then that bootloader does the work of deploying and booting the test files - this way you might not need to think about secondary media. You can simply netboot the device and chain into a different bootloader which doesn't have the restrictions living inside the firmware.
To avoid
this, we thought about serving the image through an "emulated" usb stick. On that regard, we made some progress by setting up a secondary device that would use the linux g_mass_storage module to serve the image through the USB otg port.
Our setup is then:
- The actual testing device
- The secondary device which only serves the image with g_mass_storage
- The host machine running the lava-slave application.
We thought that we could add a new device-type template to the lava server that would somehow override the deployment and boot actions to address our setup.
Please read some of the developer documentation. This isn't about overriding deployment or boot actions, this is about arranging a new list of actions in a dedicated pipeline for this board. The list would including a few new action classes written for this method.
We tried to look into the base.jinja2 file for some
sort of entry point that would allow us to, for example, run a script that would first send the image to the secondary device and secondly run the g_mass_storage module with the image file.
base.jinja2 isn't going to be enough for this. The jinja2 templates are already handled as code and this kind of work is going to need other code changes inside lava-dispatcher as well.
It sounds like you're trying to do something like what's supported via LAVA's "secondary media" support?
https://validation.linaro.org/static/docs/v2/secondary-media.html
although you might be adding more complexity to your tests that way. If your devices already boot via UEFI, could you configure them to net-boot from UEFI instead, and run your tests that way?
Netbooting from UEFI is a much simpler way of doing things. Secondary media is complex. It requires two boot operations and two deployment operations per test job - sometimes writing a rootfs to secondary media can be a lengthy process. A USB stick is not generally known for fast writes.
Cheers,
_______________________________________________ Lava-users mailing list Lava-users@lists.linaro.orgmailto:Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams ============= neil.williams@linaro.orgmailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
lava-users@lists.lavasoftware.org
-
Neil Williams -
Ros Dos Santos, Alfonso (CT RDA DS EVO OPS DIA SE 1) -
Steve McIntyre