Or add docker capability to base uboot? Like this MR: https://git.lavasoftware.org/lava/lava/-/merge_requests/1144

What do you think is the best way?

From: Larry Shen Sent: Thursday, April 23, 2020 12:16 PM To: 'lava-users@lists.lavasoftware.org' lava-users@lists.lavasoftware.org Subject: RE: Question about Docker feature for Android testing.

Additional, want to highlight it: we need to reuse the device between 2 different test methods.

From: Larry Shen Sent: Thursday, April 23, 2020 12:12 PM To: lava-users@lists.lavasoftware.orgmailto:lava-users@lists.lavasoftware.org Subject: Question about Docker feature for Android testing.

Hi, guys,

I have a question after listen Docker feature for Android testinghttps://linarotechdays.sched.com/event/ZZFc/ltd20-103-improved-android-testing-in-lava-with-docker, and the question is a little long.

1. I want to say sorry that I haven't ever tried it on android because currently not urgent for us to switch from lxc to docker. But my real question is related to this feature at least I think.

2. The whole story is: We have a device which use the "tftpboot(deploy) -> nfs(boot) -> shell(test)" mode to test, that's ok. But now we have a team which for their testcases, they need to define other logic which behavior totally different with current lava solution. There is legacy code on pc which we want to reuse, so the quicker way for us is to use docker device, in it we could do anything to control the device.

What we tried is:

- Deploy (to: docker)

- Boot (method: docker)

- Test

It's ok, as you see we could use the connection from boot in test. But as you know, not all device type accepts docker actions.

Then, I find the option after you give the presentation in linaro tech share, use docker test.

3. I tried the v2020.02 release, it's ok to use next with any device type in our job to do my things. - test: docker: image: ubuntu:18.04 Although some android related log printed like "- ANDROID_SERIAL='xxx'", but I can bear.

4. Things broken when you improve this feature in v2020.04, you add next in pipeline: WaitDeviceBoardID(self.get_board_id()

Then, the pipeline will have to wait for a udev event, but in our case, we don't have, we control device with "remote telnet device", then job hang.

5. So, my question here is: "What's the roadmap of this docker test action"? a) Is it just for android scenario? b) Why we can't make it works for more common scenario? My opinion here is: with this docker test action, we even no longer need the old "docker deploy + docker boot"? c) For this docker test, most of docker operation was written fixed in docker test, if possible to add in some place which user could configure? Like, if I want to add "-t", I can't control that although you define the interface in "DockerRun" with "def tty(self):", and others like more docker bind mounts etc?

6. BTW, a side question related to android (As definitely someday we need to switch from lxc to docker, I use this chance to ask the question). What will happened if there is adbd restart or pdu reboot in "docker test"? You add "-device" for the usb bus (I'm not sure, just think it same way as lxc), but during adbd restart or pdu reboot, the usb bus will surely changes. I didn't see docker has such kinds of ability to renew the "-device". How would that happen? Sorry again, I haven't tired, just want to know the mechanism.

7. Anyway, currently what I care about is the roadmap of "docker test". If your final direction is to make the "docker test" more common, then we are OK currently "STICK ON 2020.02". If just for android & "WaitDeviceBoardID" had to be here without any control by user, then we will give up this solution & try to find another way to reuse the device in LAVA? Your direction matters our next step!

8. Finally, Any other suggestion for our scenario which I described in item 2?

Regards, Larry

Hi, Milosz,

That's great. Then narrow down question to next:

For our scenario: a) We could now use "docker test" to meet our requirement. But still I can't set additional volume & additional flag for this container, a situation much more like the stage "docker deploy/docker boot" published in the past. At that time, we can't set them either, but now we could. Will it be improved?

b) Also, add docker callability to uboot base template also could meet my requirement.

Now, I have 2 methods available in lava now. Still, could you suggest, based on your roadmap, what's the suggested way? In fact, I'm a little lost here for why we need a new "docker test"? What's the consideration? And how a user choose?

-----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: Thursday, April 23, 2020 3:46 PM To: Larry Shen larry.shen@nxp.com Cc: lava-users@lists.lavasoftware.org Subject: [EXT] Re: [Lava-users] Question about Docker feature for Android testing.

Caution: EXT Email

On Thu, 23 Apr 2020 at 05:12, Larry Shen larry.shen@nxp.com wrote:

Hi, guys,

I have a question after listen Docker feature for Android testing, and the question is a little long.

1. I want to say sorry that I haven’t ever tried it on android because currently not urgent for us to switch from lxc to docker.

But my real question is related to this feature at least I think.

2. The whole story is:

We have a device which use the “tftpboot(deploy) -> nfs(boot) -> shell(test)” mode to test, that’s ok.

But now we have a team which for their testcases, they need to define other logic which behavior totally different with current lava solution.

There is legacy code on pc which we want to reuse, so the quicker way for us is to use docker device, in it we could do anything to control the device.

What we tried is:

•   Deploy (to: docker)
  
  

•   Boot (method: docker)
  
  

•   Test
  
  
  
  

It’s ok, as you see we could use the connection from boot in test. But as you know, not all device type accepts docker actions.

Then, I find the option after you give the presentation in linaro tech share, use docker test.

3. I tried the v2020.02 release, it’s ok to use next with any device type in our job to do my things.

• test:

  docker:

  image: ubuntu:18.04
  
  

Although some android related log printed like “- ANDROID_SERIAL='xxx'”, but I can bear.

4. Things broken when you improve this feature in v2020.04, you add next in pipeline:

WaitDeviceBoardID(self.get_board_id()

Then, the pipeline will have to wait for a udev event, but in our case, we don’t have, we control device with “remote telnet device”, then job hang.

This is a bug and it was already fixed: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.lavaso...

milosz

5. So, my question here is: “What’s the roadmap of this docker test action”?

a) Is it just for android scenario?

b) Why we can’t make it works for more common scenario?

My opinion here is: with this docker test action, we even no longer need the old “docker deploy + docker boot”?

c) For this docker test, most of docker operation was written fixed in docker test, if possible to add in some place which user could configure? Like, if I want to add “-t”, I can’t control that although you define the interface in “DockerRun” with “def tty(self):”, and others like more docker bind mounts etc?

6. BTW, a side question related to android (As definitely someday we need to switch from lxc to docker, I use this chance to ask the question).

What will happened if there is adbd restart or pdu reboot in “docker test”? You add “—device” for the usb bus (I’m not sure, just think it same way as lxc), but during adbd restart or pdu reboot, the usb bus will surely changes. I didn’t see docker has such kinds of ability to renew the “—device”. How would that happen? Sorry again, I haven’t tired, just want to know the mechanism.

7. Anyway, currently what I care about is the roadmap of “docker test”.

If your final direction is to make the “docker test” more common, then we are OK currently “STICK ON 2020.02”. If just for android & “WaitDeviceBoardID” had to be here without any control by user, then we will give up this solution & try to find another way to reuse the device in LAVA?

Your direction matters our next step!

8. Finally,

Any other suggestion for our scenario which I described in item 2?

Regards,

Larry

---

Lava-users mailing list Lava-users@lists.lavasoftware.org https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.lava...

Ok, then we may choose to use "docker deploy -> docker boot -> shell.py", and add "docker method" to base uboot templates as it looks more mature currently, thanks Milosz.

-----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: Thursday, April 23, 2020 5:05 PM To: Larry Shen larry.shen@nxp.com Cc: lava-users@lists.lavasoftware.org Subject: Re: [EXT] Re: [Lava-users] Question about Docker feature for Android testing.

Caution: EXT Email

On Thu, 23 Apr 2020 at 09:03, Larry Shen larry.shen@nxp.com wrote:

Hi, Milosz,

That's great. Then narrow down question to next:

For our scenario: a) We could now use "docker test" to meet our requirement. But still I can't set additional volume & additional flag for this container, a situation much more like the stage "docker deploy/docker boot" published in the past. At that time, we can't set them either, but now we could. Will it be improved?

Testing with 'docker' is supposed to replace LXC. Since main purpose for using LXC is testing android with adb, this is our primary focus. There are less important use cases, like hickey recovery mode. We're planning to support all of them. The main point here is that testing with 'docker' should allow to execute test shell code on the dispatcher without requiring to install stuff on the base OS. If you have additional requirements, feel free to submit patches or at least report a ticket.

b) Also, add docker callability to uboot base template also could meet my requirement.

Now, I have 2 methods available in lava now. Still, could you suggest, based on your roadmap, what's the suggested way? In fact, I'm a little lost here for why we need a new "docker test"? What's the consideration? And how a user choose?

From my pov it's pretty simple. If you need to execute some binary on dispatcher - use docker. If you plan to execute your test only on target, don't use docker.

milosz

-----Original Message----- From: Milosz Wasilewski milosz.wasilewski@linaro.org Sent: Thursday, April 23, 2020 3:46 PM To: Larry Shen larry.shen@nxp.com Cc: lava-users@lists.lavasoftware.org Subject: [EXT] Re: [Lava-users] Question about Docker feature for Android testing.

Caution: EXT Email

On Thu, 23 Apr 2020 at 05:12, Larry Shen larry.shen@nxp.com wrote:

...

Hi, guys,

I have a question after listen Docker feature for Android testing, and the question is a little long.

1. I want to say sorry that I haven’t ever tried it on android because currently not urgent for us to switch from lxc to docker.

But my real question is related to this feature at least I think.

2. The whole story is:

We have a device which use the “tftpboot(deploy) -> nfs(boot) -> shell(test)” mode to test, that’s ok.

But now we have a team which for their testcases, they need to define other logic which behavior totally different with current lava solution.

There is legacy code on pc which we want to reuse, so the quicker way for us is to use docker device, in it we could do anything to control the device.

What we tried is:

•   Deploy (to: docker)
  
  

•   Boot (method: docker)
  
  

•   Test
  
  
  
  

It’s ok, as you see we could use the connection from boot in test. But as you know, not all device type accepts docker actions.

Then, I find the option after you give the presentation in linaro tech share, use docker test.

3. I tried the v2020.02 release, it’s ok to use next with any device type in our job to do my things.

• test:

  docker:

  image: ubuntu:18.04
  
  

Although some android related log printed like “- ANDROID_SERIAL='xxx'”, but I can bear.

4. Things broken when you improve this feature in v2020.04, you add next in pipeline:

WaitDeviceBoardID(self.get_board_id()

Then, the pipeline will have to wait for a udev event, but in our case, we don’t have, we control device with “remote telnet device”, then job hang.

This is a bug and it was already fixed: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit. lavasoftware.org%2Flava%2Flava%2F-%2Fmerge_requests%2F1116&data=02 %7C01%7Clarry.shen%40nxp.com%7Ca4fb2d9d77ae4cf7097508d7e7657b7e%7C686e a1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637232295360622875&sdata=rw 1C3D4EQMZdq19sExjGfZcPBS%2FFvuHMZxymwRQUIGU%3D&reserved=0

milosz

...

5. So, my question here is: “What’s the roadmap of this docker test action”?

a) Is it just for android scenario?

b) Why we can’t make it works for more common scenario?

My opinion here is: with this docker test action, we even no longer need the old “docker deploy + docker boot”?

c) For this docker test, most of docker operation was written fixed in docker test, if possible to add in some place which user could configure? Like, if I want to add “-t”, I can’t control that although you define the interface in “DockerRun” with “def tty(self):”, and others like more docker bind mounts etc?

6. BTW, a side question related to android (As definitely someday we need to switch from lxc to docker, I use this chance to ask the question).

What will happened if there is adbd restart or pdu reboot in “docker test”? You add “—device” for the usb bus (I’m not sure, just think it same way as lxc), but during adbd restart or pdu reboot, the usb bus will surely changes. I didn’t see docker has such kinds of ability to renew the “—device”. How would that happen? Sorry again, I haven’t tired, just want to know the mechanism.

7. Anyway, currently what I care about is the roadmap of “docker test”.

If your final direction is to make the “docker test” more common, then we are OK currently “STICK ON 2020.02”. If just for android & “WaitDeviceBoardID” had to be here without any control by user, then we will give up this solution & try to find another way to reuse the device in LAVA?

Your direction matters our next step!

8. Finally,

Any other suggestion for our scenario which I described in item 2?

Regards,

Larry

---

Lava-users mailing list Lava-users@lists.lavasoftware.org https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli sts.lavasoftware.org%2Fmailman%2Flistinfo%2Flava-users&data=02%7 C01%7Clarry.shen%40nxp.com%7Ca4fb2d9d77ae4cf7097508d7e7657b7e%7C686e a1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637232295360622875&sdata= RvwJDyGrpsMn%2BWyIT6yiRLyGp3TX7N2Ds8qTXls3Mlc%3D&reserved=0