Do not use pip with LAVA. The process which starts LAVA jobs uses python-daemon and will fail if this is misconfigured. Remove everything that pip has installed on that system.
Ensure the scheduler daemon (/etc/init.d/lava_server) is using debug logging level and watch the logs to ensure it is actually running. This may have nothing to do with lava-dispatch. lava-dispatch isn't even being called - the pip install is causing the scheduler to look for the wrong support.
On 10 December 2015 at 09:01, Marc Titinger mtitinger@baylibre.com wrote:
On 09/12/2015 16:57, Neil Williams wrote:
On 9 December 2015 at 15:30, Marc Titinger mtitinger@baylibre.com wrote:
Hi Neil et al,
I'm trying to debug a simple qemu job that goes straight from running to incomplete without log creation (used to working ok, but I reinstalled everything on a different machine...)
That is the most likely problem. For completeness, I copied out your test and it ran fine here: https://staging.validation.linaro.org/scheduler/job/138097
I tried to run manually:
setsid lava-server manage schedulermonitor 14 lava-dispatch qemu0 File "/usr/lib/python2.7/dist-packages/lava_scheduler_daemon/job.py", line 226, in _run self.cancel(exc) File "/usr/lib/python2.7/dist-packages/lava_scheduler_daemon/job.py", line 157, in cancel self._protocol.transport.signalProcess(getattr(signal, signame)) exceptions.AttributeError: 'Job' object has no attribute '_protocol'
Note that I get the same issue with other jobs (boards, kvm): submission is OK, but incomplete, and no log.
In that case, it is going to be a configuration error or a mismatch in what is actually installed.
It sounds initially like you don't have the version of lava-dispatcher installed that you expect - also check that you haven't done any pip installs and that /usr/local/ has no python installs on that machine.
As a matter of fact I did some pip install consequently to hacking in lavapdu. But there is nothing that suspect under /usr/{bin,lib}/local only the lavapdu stuff. I did a pip install of daemon-1.1 though.
It's not so easy to debug through stepping, any stance you would recommend to attach or run the distpacher directly with pdb ?
Many thanks, M.