[ Related-ish to the discussion in the "Fastboot - Android boot image apply-overlay" thread, but also slightly different so I'm starting a new discussion here... ]
Hi folks,
We were just discussing in IRC about potential problems with Android images in LAVA, in terms of fitting the overlay.
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not *necessarily* a problem, except that at build time the Android builds don't know how much space we're going to need! This leads to a bit of a chicken-and-eff problem.
Anibal has already hacked around this a little bit as a workaround in a job definition - see
https://validation.linaro.org/scheduler/job/1652299/definition#defline92
In the lxc, he's converting and expanding the rootfs image to make more space, then converting back ready for flashing. This is still quite hacky - there's still no information available there about the size of the LAVA overlay. It might fit now, but what happens if there's a new LAVA version next month with (hypothetically) a much bigger overlay?
So, I'm thinking about a better way to do things here, in 2 parts:
1 .At the point when we apply the overlay, we know roughly how big it is and we could do the resizing of the image to fit directly without needing this guesswork. It's a relatively easy thing to do.
2. [Optionally] If we end up growing the rootfs image ourselves, it *might* end up overflowing the size available when we come to flash things, but "fastboot flash" will check this and abort. I even suggested that (to fail more quickly) we could parse the GPT partition table for size ourselves, but Nicolas pointed out that not all jobs will necessarily have such a partition table. Instead, *maybe* we could include a "maximum rootfs size" parameter somewhere if we care. Otherwise, just rely on fastboot doing the right thing.
Does this sound sensible?
Cheers,
On 15 November 2017 at 16:29, Steve McIntyre steve.mcintyre@linaro.org wrote:
[ Related-ish to the discussion in the "Fastboot - Android boot image apply-overlay" thread, but also slightly different so I'm starting a new discussion here... ]
Hi folks,
We were just discussing in IRC about potential problems with Android images in LAVA, in terms of fitting the overlay.
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not *necessarily* a problem, except that at build time the Android builds don't know how much space we're going to need! This leads to a bit of a chicken-and-eff problem.
Anibal has already hacked around this a little bit as a workaround in a job definition - see
https://validation.linaro.org/scheduler/job/1652299/definition#defline92
In the lxc, he's converting and expanding the rootfs image to make more space, then converting back ready for flashing. This is still quite hacky - there's still no information available there about the size of the LAVA overlay. It might fit now, but what happens if there's a new LAVA version next month with (hypothetically) a much bigger overlay?
Handling these conversions in the LXC is the best way to do this. V1 had a history of trying to be too flexible, helpful and things just get complicated and buggy precisely because of the assumptions.
So, I'm thinking about a better way to do things here, in 2 parts:
1 .At the point when we apply the overlay, we know roughly how big it is and we could do the resizing of the image to fit directly without needing this guesswork. It's a relatively easy thing to do.
Preferably inside the LXC.
- [Optionally] If we end up growing the rootfs image ourselves, it *might* end up overflowing the size available when we come to flash things, but "fastboot flash" will check this and abort. I even suggested that (to fail more quickly) we could parse the GPT partition table for size ourselves, but Nicolas pointed out that not all jobs will necessarily have such a partition table. Instead, *maybe* we could include a "maximum rootfs size" parameter somewhere if we care. Otherwise, just rely on fastboot doing the right thing.
Depends on how that maximum size is determined. If it varies by testjob based on the ptable, then it will need to be provided by the test writer - probably as a parameter to the test shell running in the LXC. If it is actually a piece of device configuration, then it will need to come from LAVA via a test shell helper - that would be in the overlay added to the LXC not the overlay which is provided for addition to whichever files are to be transferred to the device.
Does this sound sensible?
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Wed, Nov 15, 2017 at 04:43:25PM +0000, Neil Williams wrote:
On 15 November 2017 at 16:29, Steve McIntyre steve.mcintyre@linaro.org wrote:
We were just discussing in IRC about potential problems with Android images in LAVA, in terms of fitting the overlay.
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not *necessarily* a problem, except that at build time the Android builds don't know how much space we're going to need! This leads to a bit of a chicken-and-eff problem.
Anibal has already hacked around this a little bit as a workaround in a job definition - see
https://validation.linaro.org/scheduler/job/1652299/definition#defline92
In the lxc, he's converting and expanding the rootfs image to make more space, then converting back ready for flashing. This is still quite hacky - there's still no information available there about the size of the LAVA overlay. It might fit now, but what happens if there's a new LAVA version next month with (hypothetically) a much bigger overlay?
Handling these conversions in the LXC is the best way to do this. V1 had a history of trying to be too flexible, helpful and things just get complicated and buggy precisely because of the assumptions.
Nod. Maybe add a parameter for the rootfs "overlay-resize" or similar. (handwave! :-))
So, I'm thinking about a better way to do things here, in 2 parts:
1 .At the point when we apply the overlay, we know roughly how big it is and we could do the resizing of the image to fit directly without needing this guesswork. It's a relatively easy thing to do.
Preferably inside the LXC.
Yes, definitely.
2. [Optionally] If we end up growing the rootfs image ourselves, it *might* end up overflowing the size available when we come to flash things, but "fastboot flash" will check this and abort. I even suggested that (to fail more quickly) we could parse the GPT partition table for size ourselves, but Nicolas pointed out that not all jobs will necessarily have such a partition table. Instead, *maybe* we could include a "maximum rootfs size" parameter somewhere if we care. Otherwise, just rely on fastboot doing the right thing.
Depends on how that maximum size is determined. If it varies by testjob based on the ptable, then it will need to be provided by the test writer - probably as a parameter to the test shell running in the LXC. If it is actually a piece of device configuration, then it will need to come from LAVA via a test shell helper - that would be in the overlay added to the LXC not the overlay which is provided for addition to whichever files are to be transferred to the device.
Yup - we'll need to work out the exact details with the Android guys. We can leave it as a future possibility, to be done only if fastboot doesn't what we want/expect...
Cheers,
+YongQin
On 15 November 2017 at 16:29, Steve McIntyre steve.mcintyre@linaro.org wrote:
[ Related-ish to the discussion in the "Fastboot - Android boot image apply-overlay" thread, but also slightly different so I'm starting a new discussion here... ]
Hi folks,
We were just discussing in IRC about potential problems with Android images in LAVA, in terms of fitting the overlay.
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not
Steve,
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue he raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow solved?
milosz
On Tue, Nov 21, 2017 at 12:49:42PM +0000, Milosz Wasilewski wrote:
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue he raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow solved?
Hi Milosz,
The overlay we're talking about here is the LAVA test overlay with the test scripts, requested git repos etc. I'd expect it to be quite rare that you'd have compiled binaries in it, but it's possible - it depends on the particular test use cases.
Cheers,
On Tue, Nov 21, 2017 at 2:21 PM, Steve McIntyre steve.mcintyre@linaro.org wrote:
On Tue, Nov 21, 2017 at 12:49:42PM +0000, Milosz Wasilewski wrote:
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue he raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow solved?
Hi Milosz,
The overlay we're talking about here is the LAVA test overlay with the test scripts, requested git repos etc. I'd expect it to be quite rare that you'd have compiled binaries in it, but it's possible - it depends on the particular test use cases.
and the main issue is that we (e.g. the one who creates the image) does not know in advance what the size of the overlay will be, since the size is dependent on who is going to write the tests. So the ask, is for LAVA to increase the resize the image if needed.
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Tue, Nov 21, 2017 at 02:27:36PM +0100, Nicolas Dechesne wrote:
On Tue, Nov 21, 2017 at 2:21 PM, Steve McIntyre steve.mcintyre@linaro.org wrote: On Tue, Nov 21, 2017 at 12:49:42PM +0000, Milosz Wasilewski wrote:
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue he raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow solved?
The overlay we're talking about here is the LAVA test overlay with the test scripts, requested git repos etc. I'd expect it to be quite rare that you'd have compiled binaries in it, but it's possible - it depends on the particular test use cases.
and the main issue is that we (e.g. the one who creates the image) does not know in advance what the size of the overlay will be, since the size is dependent on who is going to write the tests. So the ask, is for LAVA to increase the resize the image if needed.
Nod. We're putting stories in for this today. I'll post links shortly.
Cheers,
On Tue, Nov 21, 2017 at 01:32:42PM +0000, Steve McIntyre wrote:
On Tue, Nov 21, 2017 at 02:27:36PM +0100, Nicolas Dechesne wrote:
On Tue, Nov 21, 2017 at 2:21 PM, Steve McIntyre steve.mcintyre@linaro.org wrote: On Tue, Nov 21, 2017 at 12:49:42PM +0000, Milosz Wasilewski wrote:
Ideally, the Android builds would have a rootfs configured to be as small as possible - it's automatically resized to fit the available space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay into the rootfs, space needs to be left for it to fit. That's not
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue he raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow solved?
The overlay we're talking about here is the LAVA test overlay with the test scripts, requested git repos etc. I'd expect it to be quite rare that you'd have compiled binaries in it, but it's possible - it depends on the particular test use cases.
and the main issue is that we (e.g. the one who creates the image) does not know in advance what the size of the overlay will be, since the size is dependent on who is going to write the tests. So the ask, is for LAVA to increase the resize the image if needed.
Nod. We're putting stories in for this today. I'll post links shortly.
See https://projects.linaro.org/browse/LAVA-1141 . The exact delivery date is still open to change as we prioritise work, but hopefully this captures what we need to do.
Cheers,
Hi, Steve
Sorry, I still not get the context why you need to put LAVA overlay scripts into an Android rootfs before deployed with fastboot.
Why it could not be done with adb shell after that device bootted up? Tests need to be done during the boot process?
Sounds like what we had done for juno with V1 before for lava-test-shell.
Thanks, Yongqin Liu
On 21 November 2017 at 22:47, Steve McIntyre steve.mcintyre@linaro.org wrote:
On Tue, Nov 21, 2017 at 01:32:42PM +0000, Steve McIntyre wrote:
On Tue, Nov 21, 2017 at 02:27:36PM +0100, Nicolas Dechesne wrote:
On Tue, Nov 21, 2017 at 2:21 PM, Steve McIntyre <
steve.mcintyre@linaro.org>
wrote: On Tue, Nov 21, 2017 at 12:49:42PM +0000, Milosz Wasilewski wrote:
Ideally, the Android builds would have a rootfs configured to be
as
small as possible - it's automatically resized to fit the
available
space at first boot, and leaving space otherwise is inefficient. However, if we're going to add the LAVA test overlay
into
the rootfs, space needs to be left for it to fit. That's not
Could you give a bit more context/use case on why the overlay is needed? I have similar discussions with YongQin and the main issue
he
raises is that binaries in such potential overlay might not be compatible with rootfs that is used. So I'm wondering if the overlay only contains scripts or the mentioned limitation was somehow
solved?
The overlay we're talking about here is the LAVA test overlay with
the
test scripts, requested git repos etc. I'd expect it to be quite rare that you'd have compiled binaries in it, but it's possible - it depends on the particular test use cases.
and the main issue is that we (e.g. the one who creates the image) does
not
know in advance what the size of the overlay will be, since the size is dependent on who is going to write the tests. So the ask, is for LAVA to increase the resize the image if needed.
Nod. We're putting stories in for this today. I'll post links shortly.
See https://projects.linaro.org/browse/LAVA-1141 . The exact delivery date is still open to change as we prioritise work, but hopefully this captures what we need to do.
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
On Tue, Nov 21, 2017 at 11:09:34PM +0800, Yongqin Liu wrote:
Hi, Steve
Sorry, I still not get the context why you need to put LAVA overlay scripts into an Android rootfs before deployed with fastboot.
Why it could not be done with adb shell after that device bootted up? Tests need to be done during the boot process?
Sounds like what we had done for juno with V1 before for lava-test-shell.
Apologies - I confused things by talking about an "Android rootfs". What I *really* meant to say was "rootfs for an Android device" - we're doing this with OE builds, but flashing them using fastboot onto Android devices.
Cheers,
lava-users@lists.lavasoftware.org
-
Milosz Wasilewski -
Neil Williams -
Nicolas Dechesne -
Steve McIntyre -
Yongqin Liu