Re: [Lava-users] lxc job error: Lxc container rootfs not found

Hello,

looking at the logs, it seams that the call to lxc-create failed for some reason.

Rgds

Le mar. 9 avr. 2019 à 13:29, jack lu jack.oldlu@gmail.com a écrit :

Hi, the job definition: device_type: lxc job_name: lxc-fedora timeouts: job: minutes: 30 action: minutes: 5 priority: medium visibility: public metadata: source: https://git.linaro.org/lava-team/refactoring.git path: lxc-fedora.yaml protocols: lava-lxc: name: pipeline-lxc-test distribution: fedora release: '26' arch: amd64 actions:

• deploy: timeout: minutes: 30 to: lxc
• boot: prompts:
  • '[root@(.*) /]#'
  timeout: minutes: 5 method: lxc
• test: timeout: minutes: 5 definitions:
  • repository:

http://git.linaro.org/lava-team/lava-functional-tests.git from: git path: lava-test-shell/smoke-tests-basic.yaml name: smoke-tests ================== the plain log:

• {"dt": "2019-04-09T19:01:39.876635", "lvl": "info", "msg":

"lava-dispatcher, installed at version: 2019.03+stretch"}

• {"dt": "2019-04-09T19:01:39.877171", "lvl": "info", "msg": "start: 0

validate"}

• {"dt": "2019-04-09T19:01:39.877392", "lvl": "info", "msg": "Start time:

2019-04-09 19:01:39.877373+00:00 (UTC)"}

• {"dt": "2019-04-09T19:01:39.910567", "lvl": "info", "msg": "lxc,

installed at version: 1:2.0.7-2+deb9u2"}

• {"dt": "2019-04-09T19:01:39.911883", "lvl": "info", "msg": "validate

duration: 0.03"}

• {"dt": "2019-04-09T19:01:39.912078", "lvl": "results", "msg": {"case":

"validate", "definition": "lava", "result": "pass"}}

• {"dt": "2019-04-09T19:01:39.912767", "lvl": "info", "msg": "start: 1

lxc-deploy (timeout 00:30:00) [common]"}

• {"dt": "2019-04-09T19:01:39.913192", "lvl": "debug", "msg": "start: 1.1

lxc-create-action (timeout 00:30:00) [common]"}

• {"dt": "2019-04-09T19:01:39.913481", "lvl": "debug", "msg": "nice

lxc-create -q -t download -n pipeline-lxc-test-18 -- --dist fedora --release 26 --arch amd64"}

• {"dt": "2019-04-09T19:19:10.820726", "lvl": "info", "msg": "action:

lxc-create-action\ncommand: ['nice', 'lxc-create', '-q', '-t', 'download', '-n', 'pipeline-lxc-test-18', '--', '--dist', 'fedora', '--release', '26', '--arch', 'amd64']\nmessage: Command '['nice', 'lxc-create', '-q', '-t', 'download', '-n', 'pipeline-lxc-test-18', '--', '--dist', 'fedora', '--release', '26', '--arch', 'amd64']' returned non-zero exit status 1\noutput: Command '['nice', 'lxc-create', '-q', '-t', 'download', '-n', 'pipeline-lxc-test-18', '--', '--dist', 'fedora', '--release', '26', '--arch', 'amd64']' returned non-zero exit status 1\n"}

• {"dt": "2019-04-09T19:19:10.821325", "lvl": "debug", "msg": "Container

created successfully"}

• {"dt": "2019-04-09T19:19:10.821836", "lvl": "debug", "msg": "end: 1.1

lxc-create-action (duration 00:17:31) [common]"}

• {"dt": "2019-04-09T19:19:10.822175", "lvl": "results", "msg": {"case":

"lxc-create-action", "definition": "lava", "duration": "1050.91", "extra": !!python/object/apply:collections.OrderedDict [[["status", "pipeline-lxc-test-18"]]], "level": "1.1", "namespace": "common", "result": "pass"}}

• {"dt": "2019-04-09T19:19:10.823240", "lvl": "debug", "msg": "start: 1.2

lxc-create-udev-rule-action (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.823507", "lvl": "debug", "msg": "end: 1.2

lxc-create-udev-rule-action (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:10.823928", "lvl": "debug", "msg": "start: 1.3

deploy-device-env (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.824162", "lvl": "debug", "msg": "end: 1.3

deploy-device-env (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:10.824558", "lvl": "debug", "msg": "start: 1.4

lava-overlay (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.825197", "lvl": "debug", "msg": "[common]

Preparing overlay tarball in /var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub"}

• {"dt": "2019-04-09T19:19:10.825535", "lvl": "debug", "msg": "makedir:

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin"}

• {"dt": "2019-04-09T19:19:10.825812", "lvl": "debug", "msg": "makedir:

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/tests"}

• {"dt": "2019-04-09T19:19:10.826075", "lvl": "debug", "msg": "makedir:

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/results"}

• {"dt": "2019-04-09T19:19:10.826339", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-add-keys"}

• {"dt": "2019-04-09T19:19:10.826689", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-add-sources"}

• {"dt": "2019-04-09T19:19:10.827002", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-background-process-start"}

• {"dt": "2019-04-09T19:19:10.827330", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-background-process-stop"}

• {"dt": "2019-04-09T19:19:10.827640", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-common-functions"}

• {"dt": "2019-04-09T19:19:10.827938", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-echo-ipv4"}

• {"dt": "2019-04-09T19:19:10.828244", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-install-packages"}

• {"dt": "2019-04-09T19:19:10.828554", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-installed-packages"}

• {"dt": "2019-04-09T19:19:10.828887", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-os-build"}

• {"dt": "2019-04-09T19:19:10.829215", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-probe-channel"}

• {"dt": "2019-04-09T19:19:10.829525", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-probe-ip"}

• {"dt": "2019-04-09T19:19:10.829825", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-target-ip"}

• {"dt": "2019-04-09T19:19:10.830125", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-target-mac"}

• {"dt": "2019-04-09T19:19:10.830431", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-target-storage"}

• {"dt": "2019-04-09T19:19:10.830739", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-case"}

• {"dt": "2019-04-09T19:19:10.831043", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-event"}

• {"dt": "2019-04-09T19:19:10.831371", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-feedback"}

• {"dt": "2019-04-09T19:19:10.831683", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-raise"}

• {"dt": "2019-04-09T19:19:10.831994", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-reference"}

• {"dt": "2019-04-09T19:19:10.832297", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-runner"}

• {"dt": "2019-04-09T19:19:10.832599", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-set"}

• {"dt": "2019-04-09T19:19:10.832930", "lvl": "debug", "msg": "Creating

/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/bin/lava-test-shell"}

• {"dt": "2019-04-09T19:19:10.833509", "lvl": "debug", "msg": "start:

1.4.1 lava-vland-overlay (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.833681", "lvl": "debug", "msg": "skipped

lava-vland-overlay"}

• {"dt": "2019-04-09T19:19:10.833933", "lvl": "debug", "msg": "end: 1.4.1

lava-vland-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:10.834342", "lvl": "debug", "msg": "start:

1.4.2 lava-multinode-overlay (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.834498", "lvl": "debug", "msg": "skipped

lava-multinode-overlay"}

• {"dt": "2019-04-09T19:19:10.834724", "lvl": "debug", "msg": "end: 1.4.2

lava-multinode-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:10.835161", "lvl": "debug", "msg": "start:

1.4.3 test-definition (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.835368", "lvl": "info", "msg": "Loading test

definitions"}

• {"dt": "2019-04-09T19:19:10.835832", "lvl": "debug", "msg": "start:

1.4.3.1 git-repo-action (timeout 00:12:29) [common]"}

• {"dt": "2019-04-09T19:19:10.836035", "lvl": "debug", "msg": "Using

/lava-18 at stage 0"}

• {"dt": "2019-04-09T19:19:10.836323", "lvl": "info", "msg": "Fetching

tests from http://git.linaro.org/lava-team/lava-functional-tests.git%22%7D

• {"dt": "2019-04-09T19:19:10.836547", "lvl": "debug", "msg": "Running

'/usr/bin/git clone http://git.linaro.org/lava-team/lava-functional-tests.git /var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/0/tests/0_smoke-tests --depth=1'"}

• {"dt": "2019-04-09T19:19:12.246307", "lvl": "debug", "msg": "Tests

stored (tmp) in /var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/0/tests/0_smoke-tests/lava-test-shell/smoke-tests-basic.yaml"}

• {"dt": "2019-04-09T19:19:12.249986", "lvl": "debug", "msg":

"uuid=18_1.4.3.1 testdef=None"}

• {"dt": "2019-04-09T19:19:12.250347", "lvl": "debug", "msg": "end:

1.4.3.1 git-repo-action (duration 00:00:01) [common]"}

• {"dt": "2019-04-09T19:19:12.250535", "lvl": "results", "msg": {"case":

"git-repo-action", "definition": "lava", "duration": "1.41", "extra": !!python/object/apply:collections.OrderedDict [[["commit", "0f8952f18e992371f8686c59b36f44e9c6a94a1a"], ["path", "lava-test-shell/smoke-tests-basic.yaml"], ["repository", " http://git.linaro.org/lava-team/lava-functional-tests.git"]]], "level": "1.4.3.1", "namespace": "common", "result": "pass"}}

• {"dt": "2019-04-09T19:19:12.251326", "lvl": "debug", "msg": "start:

1.4.3.2 test-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.256373", "lvl": "debug", "msg": "end:

1.4.3.2 test-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.256599", "lvl": "results", "msg": {"case":

"test-overlay", "definition": "lava", "duration": "0.01", "extra": !!python/object/apply:collections.OrderedDict [[["name", "smoke-tests"], ["from", "git"], ["uuid", "18_1.4.3.1"], ["path", "lava-test-shell/smoke-tests-basic.yaml"], ["repository", " http://git.linaro.org/lava-team/lava-functional-tests.git"]]], "level": "1.4.3.2", "namespace": "common", "result": "pass"}}

• {"dt": "2019-04-09T19:19:12.257266", "lvl": "debug", "msg": "start:

1.4.3.3 test-install-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.264770", "lvl": "debug", "msg": "end:

1.4.3.3 test-install-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.264982", "lvl": "results", "msg": {"case":

"test-install-overlay", "definition": "lava", "duration": "0.01", "extra": !!python/object/apply:collections.OrderedDict [[["name", "smoke-tests"], ["from", "git"], ["uuid", "18_1.4.3.1"], ["path", "lava-test-shell/smoke-tests-basic.yaml"], ["repository", " http://git.linaro.org/lava-team/lava-functional-tests.git"], ["skipped test-install-overlay", "18_1.4.3.1"]]], "level": "1.4.3.3", "namespace": "common", "result": "pass"}}

• {"dt": "2019-04-09T19:19:12.265667", "lvl": "debug", "msg": "start:

1.4.3.4 test-runscript-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.288100", "lvl": "debug", "msg": "runner

path: /var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/0/tests/0_smoke-tests test_uuid 18_1.4.3.1"}

• {"dt": "2019-04-09T19:19:12.288888", "lvl": "debug", "msg": "end:

1.4.3.4 test-runscript-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.289115", "lvl": "results", "msg": {"case":

"test-runscript-overlay", "definition": "lava", "duration": "0.02", "extra": !!python/object/apply:collections.OrderedDict [[["name", "smoke-tests"], ["from", "git"], ["uuid", "18_1.4.3.1"], ["path", "lava-test-shell/smoke-tests-basic.yaml"], ["repository", " http://git.linaro.org/lava-team/lava-functional-tests.git"], ["filename", "/var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/0/tests/0_smoke-tests/run.sh"]]], "level": "1.4.3.4", "namespace": "common", "result": "pass"}}

• {"dt": "2019-04-09T19:19:12.289613", "lvl": "info", "msg": "Creating

lava-test-runner.conf files"}

• {"dt": "2019-04-09T19:19:12.289801", "lvl": "debug", "msg": "Using

lava-test-runner path: /var/lib/lava/dispatcher/tmp/18/lava-overlay-6drrbbub/lava-18/0 for stage 0"}

• {"dt": "2019-04-09T19:19:12.290062", "lvl": "debug", "msg": "-

0_smoke-tests"}

• {"dt": "2019-04-09T19:19:12.290340", "lvl": "debug", "msg": "end: 1.4.3

test-definition (duration 00:00:01) [common]"}

• {"dt": "2019-04-09T19:19:12.290775", "lvl": "debug", "msg": "start:

1.4.4 compress-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.335451", "lvl": "debug", "msg": "end: 1.4.4

compress-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.335979", "lvl": "debug", "msg": "start:

1.4.5 persistent-nfs-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.336202", "lvl": "debug", "msg": "end: 1.4.5

persistent-nfs-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.336450", "lvl": "debug", "msg": "end: 1.4

lava-overlay (duration 00:00:02) [common]"}

• {"dt": "2019-04-09T19:19:12.336825", "lvl": "debug", "msg": "start: 1.5

apply-lxc-overlay (timeout 00:12:28) [common]"}

• {"dt": "2019-04-09T19:19:12.337042", "lvl": "debug", "msg":

"@ApplyLxcOverlay run,lxc_default_path: /var/lib/lxc"}

• {"dt": "2019-04-09T19:19:12.337208", "lvl": "debug", "msg":

"@..lxc_rootfs_path: /var/lib/lxc/pipeline-lxc-test-18/rootfs"}

• {"dt": "2019-04-09T19:19:12.342011", "lvl": "exception", "msg": "Lxc

container rootfs not found"}

• {"dt": "2019-04-09T19:19:12.342357", "lvl": "debug", "msg": "end: 1.5

apply-lxc-overlay (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.342647", "lvl": "results", "msg": {"case":

"apply-lxc-overlay", "definition": "lava", "duration": "0.01", "extra": !!python/object/apply:collections.OrderedDict [[["fail", "Lxc container rootfs not found"]]], "level": "1.5", "namespace": "common", "result": "fail"}}

• {"dt": "2019-04-09T19:19:12.343165", "lvl": "exception", "msg": "Lxc

container rootfs not found"}

• {"dt": "2019-04-09T19:19:12.343439", "lvl": "info", "msg": "end: 1

lxc-deploy (duration 00:17:32) [common]"}

• {"dt": "2019-04-09T19:19:12.343719", "lvl": "results", "msg": {"case":

"lxc-deploy", "definition": "lava", "duration": "1052.43", "extra": !!python/object/apply:collections.OrderedDict [[["fail", "Lxc container rootfs not found"]]], "level": "1", "namespace": "common", "result": "fail"}}

• {"dt": "2019-04-09T19:19:12.344246", "lvl": "info", "msg": "Cleaning

after the job"}

• {"dt": "2019-04-09T19:19:12.344907", "lvl": "debug", "msg": "start: 4.1

power-off (timeout 00:00:10) [common]"}

• {"dt": "2019-04-09T19:19:12.345210", "lvl": "debug", "msg": "end: 4.1

power-off (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.345768", "lvl": "debug", "msg": "start: 4.2

read-feedback (timeout 00:05:00) [common]"}

• {"dt": "2019-04-09T19:19:12.346026", "lvl": "debug", "msg": "No

connection for namespace common"}

• {"dt": "2019-04-09T19:19:12.346318", "lvl": "debug", "msg": "end: 4.2

read-feedback (duration 00:00:00) [common]"}

• {"dt": "2019-04-09T19:19:12.346576", "lvl": "debug", "msg": "lava-lxc

protocol: issue stop"}

• {"dt": "2019-04-09T19:19:12.346814", "lvl": "debug", "msg": "lava-lxc

protocol: executing 'lxc-stop -n pipeline-lxc-test-18 -k'"}

• {"dt": "2019-04-09T19:19:12.353290", "lvl": "debug", "msg": "lava-lxc

protocol: FAILED executing 'lxc-stop -n pipeline-lxc-test-18 -k'"}

• {"dt": "2019-04-09T19:19:12.353922", "lvl": "debug", "msg": "lava-lxc

protocol: issue destroy"}

• {"dt": "2019-04-09T19:19:12.354214", "lvl": "debug", "msg": "lava-lxc

protocol: executing 'lxc-destroy -n pipeline-lxc-test-18 -f'"}

• {"dt": "2019-04-09T19:19:12.360435", "lvl": "debug", "msg": "lava-lxc

protocol: FAILED executing 'lxc-destroy -n pipeline-lxc-test-18 -f'"}

• {"dt": "2019-04-09T19:19:12.361043", "lvl": "debug", "msg": "lava-lxc

protocol: executing 'udevadm control --reload-rules'"}

• {"dt": "2019-04-09T19:19:12.364938", "lvl": "debug", "msg": "lava-lxc

protocol finalised."}

• {"dt": "2019-04-09T19:19:12.365644", "lvl": "info", "msg": "Root tmp

directory removed at /var/lib/lava/dispatcher/tmp/18"}

• {"dt": "2019-04-09T19:19:12.370953", "lvl": "error", "msg": "LAVABug:

This is probably a bug in LAVA, please report it."}

• {"dt": "2019-04-09T19:19:12.371234", "lvl": "results", "msg": {"case":

"job", "definition": "lava", "error_msg": "Lxc container rootfs not found", "error_type": "Bug", "result": "fail"}}

Remi Duraffort remi.duraffort@linaro.org 于2019年4月9日周二 下午6:45写道：

...

Hello,

could you paste the raw logs instead of just one line? Please, also add the job definition.

Rgds

Le lun. 8 avr. 2019 à 12:11, jack lu jack.oldlu@gmail.com a écrit :

...

Hi,according https://validation.linaro.org/static/docs/v2/deploy-lxc.html?highlight=lxc.

I run a sample lxc Job Definition as above.. the error msg : Lxc container rootfs not found _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users

-- Rémi Duraffort LAVA Team, Linaro