Hello Neil,

This is what I indeed needed from you. Since I do lack/I am very ignorant of what Lava does?!

It perfectly makes sense, now. Yes, I see. You need to unpack the whole initramfs, then to take environment variables - env.dut.yaml and another (?) stuff (some preparation scripts etc., maybe to add some necessary debian packages) which I can just guess.

This does explain why mine and your offsets vary so much, since you probably for your tests put inside much more stuff, me just an environment... :-)

I have just one more question: how I can, upon finished tests, to export the whole initramfs outside the Lava test unit? I might need to store the results on ramdiskfs, and then to export them out (before the PDU goes off). Or, perhaps, I can do this with some mount commands' tricks?

I can also do this with NFS, but I prefer earlier (since this is true independent DUT test).

Any simplistic script which does this?

I would like to thank you for detailed explanation,
Zoran
_______

On Thu, Mar 22, 2018 at 11:27 AM, Neil Williams <neil.williams@linaro.org> wrote:
On 22 March 2018 at 07:59, Zoran S <zoran.stojsavljevic.de@gmail.com> wrote:
Hello Neil,

Actually, you have the exact SAME problem as I do. Even worse... ;-)

You are misinterpreting what LAVA is doing. The ramdisk has to be decompressed, modified and recompressed, possibly then adding a U-Boot header. The modification will increase the size of the ramdisk because it adds all of the test shell definitions. The filename is entirely arbitrary and we use a consistent internal name just for convenience and debugging. Test definitions will change over time and so the final size will differ between test jobs.
 

Now, please, inspect the following:

In this example, you have the following as download:
[1] kernel:
ownloading http://images.validation.linaro.org/snapshots.linaro.org/components/lava/standard/debian/jessie/armhf/4/vmlinuz saving as /var/lib/lava/dispatcher/tmp/214334/tftp-deploy-BxFGlh/kernel/vmlinuz total size: 3187808 (3MB)
.../tmp/214334/tftp-deploy-BxFGlh/kernel/vmlinuz

And actual HW kernel tftp:
Bytes transferred = 3187808 (30a460 hex)
.../214334/tftp-deploy-BxFGlh/kernel/vmlinuz

Whilst for the ramdiskfs:
downloading http://images.validation.linaro.org/snapshots.linaro.org/components/lava/standard/debian/jessie/armhf/4/initramfs.cpio.gz saving as /var/lib/lava/dispatcher/tmp/214334/tftp-deploy-BxFGlh/ramdisk/initramfs.cpio.gz total size: 12427620 (11MB)
.../tmp/214334/tftp-deploy-BxFGlh/ramdisk/initramfs.cpio.gz

And actual HW ramdisk tftp:
Bytes transferred = 35072561 (2172a31 hex)
.../214334/tftp-deploy-BxFGlh/ramdisk/ramdisk.cpio.gz.uboot

Wow! Almost three times (3x) in size (and actual names differ)???

The same here:
And so on...
_______

Nope. Your arguments are not bought. You need to come with 100% logical explanation, not some hocus-pocus. And you need exactly to explain the following:

[1] Why the HW tftp of ramdiskfs differs MORE that 64 bytes from original one, since your U-Boot header script exactly adds 64 bytes, which is the correct behaviour?!

[common] Applying overlay /var/lib/lava/dispatcher/tmp/214194/compress-overlay-Uq4NXM/overlay-1.5.2.5.tar.gz to ramdisk
 

Please, do note that this all works perfectly as when I use just plain shells. Everything behaves logically. U-Boot always does the correct tftp of all the ingredients (uImage, initramfs, .tdb files)! So the local environment Lava is in is VERY correct.

[2] Why it ONLY happens to initramfs, NOT to other components, which are downloaded/tftp-ed correctly in Lava?

Because the test shell definitions need to operate within a shell and only the ramdisk provides that, not the kernel, modules.tar or dtb.
 

[3] Why EVERY time the HW tftp of the length of initramfs differs (it is NOT a constant length >> 64, it is a variable length)???

Because the files have been modified inside the ramdisk with contents which vary over time.
 

[4] Why the names, after adding "64byte" header differ?

To make our lives easier in the code and because the name of the file has no impact on what gets delivered over TFTP. It makes things much clearer when decompressing, unpacking, modifying, repacking and compressing if the filename is tightly controlled.
 

I hope my remarks will help you to debug Lava problems, and to explain such a buggy behaviour.

It is not buggy. If you simply want to run the ramdisk unchanged, you can set apply-overlay: false and do a simple boot test.

Alternatively, use NFS.
 

(I kinda envision what the problem might be, but I'll wait for you all, Lava R&D, for the official explanation)

Thank you,
Zoran
_______

On Wed, Mar 21, 2018 at 5:06 PM, Neil Williams <neil.williams@linaro.org> wrote:
On 21 March 2018 at 15:58, Zoran S <zoran.stojsavljevic.de@gmail.com> wrote:
I added sha256 into the job definition as you requested:

    ramdisk:
      image_arg: -drive format=raw,file={rootfs}
      url: http://localhost:8010/initramfs/initramfs.cpio.gz
      sha256sum: 2f3c091403548c05e79c25d602d631fbcc58af68a9266b78e43a97b0ded670d5
      compression: gz
      # the bootloader needs a u-boot header on the modified ramdisk
      add-header: u-boot
    # despite this being a Debian initramfs, it is not a complete Debian rootfs, so use oe compatibility

And the downloading size is always the same:

downloading http://localhost:8010/initramfs/initramfs.cpio.gzsaving as /var/lib/lava/dispatcher/tmp/121/tftp-deploy-GmW9Hh/ramdisk/initramfs.cpio.gztotal size: 46275637 (44MB)

downloading http://localhost:8010/initramfs/initramfs.cpio.gz saving as /var/lib/lava/dispatcher/tmp/120/tftp-deploy-Lej43g/ramdisk/initramfs.cpio.gz total size: 46275637 (44MB)

downloading http://localhost:8010/initramfs/initramfs.cpio.gz saving as /var/lib/lava/dispatcher/tmp/119/tftp-deploy-APtybp/ramdisk/initramfs.cpio.gz total size: 46275637 (44MB)

And nothing really happens, everything stays the same, repeats!?

Actually, the whole job fails here:

beaglebone login: #
#
ShellCommand command timed out.: Sending # in case of corruption. Connection timeout 00:00:47, retry in 00:00:05
pattern: ['-+\\[ cut here \\]-+\\s+(.*\\s+-+\\[ end trace (\\w*) \\]-+)', '(Unhandled fault.*)\r\n', 'Kernel panic - (.*) end Kernel panic', 'Stack:\\s+(.*\\s+-+\\[ end trace (\\w*) \\]-+)', 'ALERT! .* does not exist.\\s+Dropping to a shell!', '\\(initramfs\\)', 'Login timed out', 'Login incorrect']
#
Password: #
Login incorrect
Matched prompt #7: Login incorrect
auto_login not enabled but image requested login details.

Since the ramdiskfs id corrupted, it is obvious.

From that snippet of log it is not at all obvious. 

Attach the complete test job log, the device dictionary and the test job submission in any reply.

As previously advised, please change to using a U-Boot build from the Debian uboot packages and then use the gold standard test images and test job submissions from the documentation. You can put the original U-Boot build back into place afterwards but you need to get to a point where you have a known working reference. Then make one change at a time until you learn about how the system has to work.
 

Can I add somehow in devices in my bbb01.jinja2 login and password (login root, password <empty>)? For the testing purposes?

That would be part of the test job. 

 

What would be the command?

What could be cause to this effect???

Thank you,
Zoran
_______

On Wed, Mar 21, 2018 at 3:16 PM, Neil Williams <neil.williams@linaro.org> wrote:
On 21 March 2018 at 13:21, Zoran S <zoran.stojsavljevic.de@gmail.com> wrote:
Hello Folks,

I have really interesting problem with Lava, this time while executing tests.

Lava versions:
root@stretch:/usr/share# dpkg -l lava-server lava-dispatcher
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                       Version            Architecture       Description
+++-==========================-==================-==================-=========================================================
ii  lava-dispatcher            2018.2.post2-1+str amd64
Linaro Automated Validation Architecture dispatcher
ii  lava-server                2018.2-1+stretch   all
Linaro Automated Validation Architecture server
root@stretch:/usr/share#
_______

This problem appears while downloading initramfs. Although the
initramfs I am using from localhost://8010 has the fixed size:
46275637 (decimal), while I download ramdisk.cpio.gz.uboot
(which in theory should be 64 bytes more, exactly 46275701):

I am getting the following on The Same Jobs, which I repeated in Lava:
Job #113: 46597208 (decimal)
Job #114: 46596349 (decimal)
Job #115: 46595788 (decimal)

Downloading automatically calculates the checksum of the downloaded file. e.g.


In each case, identical files produce identical checksums.

You should add the checksum to the test job submission so that LAVA can validate that the file being downloaded is always the same.


In our tests, the same file is always the same download size:
 

In other words, I am downloading each time The Same ingredients:

http://localhost:8010/initramfs/initramfs.cpio.gz
http://localhost:8010/cip-example/cip_v4.4.120-cyclic/v4.4.120-cip20-rt13/arm/omap2plus_defconfig/zImage
http://localhost:8010/cip-example/cip_v4.4.120-cyclic/v4.4.120-cip20-rt13/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb

Where I have all three time exact size for:
zImage - 4167704 (3f9818 hex)
am335x-boneblack.dtb - 31552 (7b40 hex)

I removed u-boot-tools, then I installed it back. But this did not
help. service tftpd-hpa restart did not help as well.

So, Iwill continue investigating this problem myself, but did amybody
notice the same?

Sounds like a local problem. Use the checksum support.
 

Thank you,
Zoran
_______________________________________________
Lava-users mailing list
Lava-users@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lava-users



--

Neil Williams
=============
neil.williams@linaro.org
http://www.linux.codehelp.co.uk/