Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava? Let's say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
Best regards,
Denis
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Thanks Milosz,
Do you have any idea on how intrusive Lava is on such boot processes? Based on what I see, without any further analysis, boot time is much higher when measured inside Lava. In my example, I measure Linux + Userland boot time, and I get about 20s more using Lava. As I said, I haven't analyzed yet.
But, globally speaking, we have noticed so far that time measurements, benchmarkings, when performed via Lava, produce results significantly different than when performed outside Lava. Boot time is the latest example. I wonder if it is due to Lava (Lava may not be designed for this), or maybe is it due to our setup.
Regards,
Denis -----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: mardi 21 août 2018 11:58 To: Denis HUMEAU denis.humeau@st.com Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 22 August 2018 at 08:02, Denis HUMEAU denis.humeau@st.com wrote:
Thanks Milosz,
Do you have any idea on how intrusive Lava is on such boot processes? Based on what I see, without any further analysis, boot time is much higher when measured inside Lava. In my example, I measure Linux + Userland boot time, and I get about 20s more using Lava. As I said, I haven't analyzed yet.
I'm suffering from the same problem. The time recorded in auto-login-action might be affected by some actions LAVA performs on serial. Recently there was 'kernel-messages' test result introduced. It might give better match. But I guess we'll need to wait for some core LAVA folks to comment.
milosz
But, globally speaking, we have noticed so far that time measurements, benchmarkings, when performed via Lava, produce results significantly different than when performed outside Lava. Boot time is the latest example. I wonder if it is due to Lava (Lava may not be designed for this), or maybe is it due to our setup.
Regards,
Denis -----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: mardi 21 août 2018 11:58 To: Denis HUMEAU denis.humeau@st.com Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 22 August 2018 at 10:46, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 22 August 2018 at 08:02, Denis HUMEAU denis.humeau@st.com wrote:
Thanks Milosz,
Do you have any idea on how intrusive Lava is on such boot processes? Based on what I see, without any further analysis, boot time is much higher when measured inside Lava. In my example, I measure Linux + Userland boot time, and I get about 20s more using Lava. As I said, I haven't analyzed yet.
I'm suffering from the same problem. The time recorded in auto-login-action might be affected by some actions LAVA performs on serial. Recently there was 'kernel-messages' test result introduced. It might give better match. But I guess we'll need to wait for some core LAVA folks to comment.
Don't know if there's anything LAVA-specific that disrupts boot time measurement, but when I have needed to track boot times I have always had to disable the serial console to get meaningful numbers. In most ARM boards, it can slow boot down by several seconds, depending on the amount of output.
Regards,
Tomeu
milosz
But, globally speaking, we have noticed so far that time measurements, benchmarkings, when performed via Lava, produce results significantly different than when performed outside Lava. Boot time is the latest example. I wonder if it is due to Lava (Lava may not be designed for this), or maybe is it due to our setup.
Regards,
Denis -----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: mardi 21 août 2018 11:58 To: Denis HUMEAU denis.humeau@st.com Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Hello Tomeo,
Sorry for this basic question, but how do you disable the serial console and still manage to run with Lava? I assume that serial console is mandatory for Lava to work.
Regards,
Denis
-----Original Message----- From: Tomeu Vizoso tomeu@tomeuvizoso.net Sent: mercredi 22 août 2018 12:53 To: Milosz Wasilewski milosz.wasilewski@linaro.org Cc: Denis HUMEAU denis.humeau@st.com; lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 22 August 2018 at 10:46, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 22 August 2018 at 08:02, Denis HUMEAU denis.humeau@st.com wrote:
Thanks Milosz,
Do you have any idea on how intrusive Lava is on such boot processes? Based on what I see, without any further analysis, boot time is much higher when measured inside Lava. In my example, I measure Linux + Userland boot time, and I get about 20s more using Lava. As I said, I haven't analyzed yet.
I'm suffering from the same problem. The time recorded in auto-login-action might be affected by some actions LAVA performs on serial. Recently there was 'kernel-messages' test result introduced. It might give better match. But I guess we'll need to wait for some core LAVA folks to comment.
Don't know if there's anything LAVA-specific that disrupts boot time measurement, but when I have needed to track boot times I have always had to disable the serial console to get meaningful numbers. In most ARM boards, it can slow boot down by several seconds, depending on the amount of output.
Regards,
Tomeu
milosz
But, globally speaking, we have noticed so far that time measurements, benchmarkings, when performed via Lava, produce results significantly different than when performed outside Lava. Boot time is the latest example. I wonder if it is due to Lava (Lava may not be designed for this), or maybe is it due to our setup.
Regards,
Denis -----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: mardi 21 août 2018 11:58 To: Denis HUMEAU denis.humeau@st.com Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Tue, 28 Aug 2018 at 08:11, Denis HUMEAU denis.humeau@st.com wrote:
Hello Tomeo,
Sorry for this basic question, but how do you disable the serial console and still manage to run with Lava?
You can't disable the serial console entirely, what you can do is set the kernel command line to quiet (but this will also hide a lot of useful output). I would recommend using specific test job definitions which only do a boot test when running with a quiet kernel and also ensure that the actual kernel image correctly boots (without quiet) before running the measurement test job. Equally, if what you are testing is kernel boot time, be aware that you should only use ramdisk test jobs and use as small a ramdisk as possible so that the INIT stage happens with as little fuss as possible. As it's only a kernel boot test, for example, you do not want INIT trying to get a DHCP address when what you want is a time measurement. Different test jobs for different use cases. You certainly don't want the extra time delay of communicating with NFS - that delay is not predictable and the rest of INIT will be affected if the LAN is slow. Try to get a cusomised ramdisk which (as near as possible) always takes the same amount of time to run INIT.
The timing also depends on the DUT and the kernel configuration. If the kernel is very noisy and produces a lot of debug to the console, it can take some time for the DUT to send that information and LAVA cannot do any measurements until the output has been sent from the DUT over serial. It is lava-run which is doing the timing, on the worker, so it is as close as we can get to the DUT.
The final option is to write a custom kernel module / kernel configuration which measures and records the kernel boot time independently of the serial port. Then access that value after boot from a Lava Test Shell. This gives you full control over whether you are measuring only the kernel boot time or kernel + INIT.
I assume that serial console is mandatory for Lava to work.
Regards,
Denis
-----Original Message----- From: Tomeu Vizoso tomeu@tomeuvizoso.net Sent: mercredi 22 août 2018 12:53 To: Milosz Wasilewski milosz.wasilewski@linaro.org Cc: Denis HUMEAU denis.humeau@st.com; lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 22 August 2018 at 10:46, Milosz Wasilewski < milosz.wasilewski@linaro.org> wrote:
On 22 August 2018 at 08:02, Denis HUMEAU denis.humeau@st.com wrote:
Thanks Milosz,
Do you have any idea on how intrusive Lava is on such boot processes?
Based on what I see, without any further analysis, boot time is much higher when measured inside Lava. In my example, I measure Linux + Userland boot time, and I get about 20s more using Lava.
As I said, I haven't analyzed yet.
I'm suffering from the same problem. The time recorded in auto-login-action might be affected by some actions LAVA performs on serial. Recently there was 'kernel-messages' test result introduced. It might give better match. But I guess we'll need to wait for some core LAVA folks to comment.
Don't know if there's anything LAVA-specific that disrupts boot time measurement, but when I have needed to track boot times I have always had to disable the serial console to get meaningful numbers. In most ARM boards, it can slow boot down by several seconds, depending on the amount of output.
Regards,
Tomeu
milosz
But, globally speaking, we have noticed so far that time measurements,
benchmarkings, when performed via Lava, produce results significantly different than when performed outside Lava. Boot time is the latest example. I wonder if it is due to Lava (Lava may not be designed for this), or maybe is it due to our setup.
Regards,
Denis -----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: mardi 21 août 2018 11:58 To: Denis HUMEAU denis.humeau@st.com Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] Boot time measurement with Lava
On 21 August 2018 at 10:56, Denis HUMEAU denis.humeau@st.com wrote:
Hello Lava users,
Do you have any example or advice to share on a way to measure boot time using Lava?
Let’s say that I use u-boot, a minimal boot for a device running Linux, and I want to measure the time spent between Power On (pdu-reboot e.g) and Linux prompt.
You can find it in the 'lava' suite results as 'auto-login-action'. It
has a measurement which is roughly what you expect.
milosz
Best regards,
Denis
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
lava-users@lists.lavasoftware.org
-
Denis HUMEAU -
Milosz Wasilewski -
Neil Williams -
Tomeu Vizoso