This change could be attempted in compression.py, directly? I am currently trying anything I can think of in order to get past this.
-----Original Message----- From: Steve McIntyre [mailto:steve.mcintyre@linaro.org] Sent: Tuesday, December 19, 2017 5:53 PM To: Dragos Iorgulescu Dragos.Iorgulescu@enea.com Cc: senthil.kumaran@linaro.org; Remi Duraffort remi.duraffort@linaro.org; lava-users@lists.linaro.org Subject: Re: [Lava-users] Unpacking tar.bz2 root filesystems seems to malfunction
On Tue, Dec 19, 2017 at 03:29:46PM +0000, Dragos Iorgulescu wrote:
Is it possible to use another compression format, in order to bypass this? I just tried doing that and used the rootfs tar repacked as a gzip file. Even if if specify "compression: gz" in the job definition, it appears that LAVA expects this rootfs to be a bzip2 archive...
https://paste.debian.net/1001483/ -> plain log https://paste.debian.net/1001485/ -> job definition
Mind you that specifying the "archive: tar" attribute doesn't make a difference (tried that already)
Right. We're using the python tarfile module, and that's where the error is coming from, e.g. in line 84 of your log. That's from lava_dispatcher/utils/compression.py:untar_file():
except tarfile.TarError as exc: raise JobError("Unable to unpack %s: %s" % (infile, str(exc)))
the tarfile module includes built-in support for gz and bz2 compression and decompression, and by default will attempt to auto-decompress things transparently. Maybe it's being confused by something in your rootfs. It's not double-compressed (or similar?), I hope? You're referencing a file: url so I can't grab it to take a look here.
Maybe we should be tarfile.open() with mode='r:' to turn off the auto-decompress here, as we've already decompressed it during the download...
Cheers,