On 20 February 2018 at 16:50, Tim Jaacks <tim.jaacks@garz-fricke.com> wrote:

Hi Neil, thanks for your reply.

>Why do you need an in-house bootloader? (Hint: Just because you can / have
>is the worst possible reason.) Honestly, just find some other way to do it
>- it will likely be much quicker.

We have developed our own bootloader for our hardware, for various reasons, and want our OS to be tested in combination with it. Our devices do not support any other bootloader except our own anyway, so there is no other possibility. This is what we have, and we need to automate testing somehow. So I am trying to figure out whether LAVA is suitable for this task.

LAVA can do the work, probably yes. I can't say for sure without seeing the final changeset.

>Supporting a new bootloader is a very complex and time-consuming process
>for any test framework. It typically takes several months. Once done, there
>is an ongoing maintenance burden as the code for the supported bootloaders
>continues to advance.

This is not a problem generally. We are planning long-term and want a future-proof solution.

Future proofing means keeping up with upstream and, hopefully, contributing the support back to upstream. A new Strategy class is a requirement for inclusion upstream, to be able to isolate this bootloader from future changes to better support U-Boot in ways that your bootloader cannot manage.

LAVA is generic and requires high reliability from test instances. The way to maintain that long term is to ensure proper isolation between Strategy classes so that support for, in this case, U-Boot is never held back by missing support in your in-house bootloader. (Or more likely, future changes in our U-Boot support would simply disable your bootloader support because your bootloader is not actually U-Boot.) There is also the problem of testing the support upstream - we have lots of U-Boot devices but your bootloader will need to be tested with it's own sample jobs and unit test support.

Whilst it can be a non-trivial effort, in this case it might not take as long as it has typically taken in the past. I can't be certain at this stage, so best to be cautious and go with the average expectation of at least a few months, likely several.

>You need a new Strategy Class, as per the documentation. You must not
>overload the "method: u-boot" support for anything except U-Boot. You must
>create a new boot method.

Why? What is wrong with using the u-boot method for a bootloader which works similarly?

You can inherit from the common classes - generally named BootloaderCommon* - but you must not just modify the UBoot classes or use the u-boot syntax in submissions to something which is not U-Boot. Just the same as with Grub or UEFI or fastboot support. Your test job syntax and device configuration needs to sit alongside U-Boot without interfering with existing U-Boot support. We have a lot of U-Boot devices to support and we cannot allow future changes to U-Boot to held hostage to changes in a must less well supported similar-but-not-the-same bootloader.

Your bootloader is not U-Boot, it cannot pretend to be U-Boot, that is not future proof. It needs a new Strategy Class which can re-use some common code, just not use the U-Boot classes directly. That is the only way to ensure that the support for your internal bootloader continues to operate as the U-Boot support continues to develop. This is the cost of writing another bootloader instead of using something which is already supported, like U-Boot.

Things like enhancements to tighten up the bootloader error detection code to catch more and more U-Boot specific error messages simply won't work with your bootloader. These are necessary changes and we need to be able to rely on predictable behaviour.

As far as I can see right now, I would only remove stuff from the u-boot method and set a bunch of already existing variables to new values. I do not see the need for a completely new boot method. Is there any hard reason for this, except that it would be neater?

It would leave you with an unmaintainable fork as this would not be acceptable upstream. This isn't about being neat, this is about being generic and retaining support for a much more popular and more widely supported bootloader. Someone, at some point, is going to require support for a U-Boot device alongside these devices and you'll need to have the separation to do that.

Mit freundlichen Grüßen / Best regards
Tim Jaacks
DEVELOPMENT ENGINEER
Garz & Fricke GmbH
Tempowerkring 2
21079 Hamburg
Direct: +49 40 791 899 - 55
Fax: +49 40 791899 - 39
tim.jaacks@garz-fricke.com
www.garz-fricke.com
SOLUTIONS THAT COMPLETE!

Sitz der Gesellschaft: D-21079 Hamburg
Registergericht: Amtsgericht Hamburg, HRB 60514
Geschäftsführer: Matthias Fricke, Manfred Garz

Neil Williams
=============
neil.williams@linaro.org
http://www.linux.codehelp.co.uk/