Hi Lava Users,

After upgrading to LAVA 2021.01, we are facing an issue of jobs not completing/exiting (once submitted) and getting stuck in “Running” state, doesn’t matter if the job is successful or unsuccessful/incomplete/fail. After sometime if we try to cancel the job it gets stuck in “Cancelling” state. Due to this the device gets blocked and subsequent jobs are sent to “Scheduled” state.

Now only possible solution to free up the device, is to delete the job from LAVA administration

 LAVA Administration > Test jobs (under lava_scheduler_app) > {job id} > Delete .

Though this way the device is recovered but the whole job gets deleted.

I have tried with different boot and deploy methods like nfs, uuu, ums, and minimal. In all cases I am getting similar issue.

This issue is only observed while using imx devices.

Ssh based jobs are working properly and also other x86 based devices while booting through minimal method.

I have attached screenshot of the jobs (Lava-job-begin.PNG and Lava-job-end.PNG).

What can be the issue here ?

Thanks, 

Bhargav