Hello Neil,

I found the cause why the whole Lava behaves insane. It is that VM
Virtual Box stretch.vmdk reached the limit (device full).

Since stretch.vmdk is fixed size file, I've managed to clone and
create stretch.vdi (dynamic Virtual disk), which is 4x size of stretch
.vmdk. So far the new (cloned) VM, till Lava (and there are tons of SW
running prior Lava) behave very correctly.

/dev/sda1       38240460 8160436  28118092  23% /

I have tried again to run the scheduler, with qemu01 which perfectly
worked before. It behaves the same. The job (qemu) stays indefinitely
in submitted state.

This job (qemu) should be NOT dependent of anything else, correct?

If
it is (of other independent jobs), the whole Lava project is created
as wrong architecture?!

It is very clear to me that Lava does not behave correctly. It is
Lava's fault, I am sure.

Please, provide the way to reset the whole Lava in some synch-ed way.

Commands service lava-server restart and similar do not work.

Thank you,
Zoran
_______

On Wed, Feb 21, 2018 at 3:05 PM, Neil Williams <neil.williams@linaro.org> wrote:
> On 21 February 2018 at 13:58, Zoran S <zoran.stojsavljevic.de@gmail.com>
> wrote:
>>
>> Hello Neil,
>>
>> This does not help me at all. I ran qemu01 device, which was perfectly
>> OK. Now it is running, and stuck in running state. I cancelled this
>> job, but it is stucked in cancelling state!?
>
>
>>
>> I need method to reset the whole LAVA.
>
>
> Not necessarily. You can do that but you are better off learning about the
> problems and the causes. LAVA is not a small utility that you reset at the
> first sign of problems. Everyone installing LAVA needs some level of
> administrative skills and that can involve a learning curve. apt-get install
> is only the very beginning of the work.
>
>>
>>
>> > What version of LAVA are you running? There were important changes in
>> > scheduling after the removal of V1 in 2018.1
>>
>> root@stretch:/etc/lava-server/dispatcher-config/devices# dpkg -l
>> lava-server lava-dispatcher
>> Desired=Unknown/Install/Remove/Purge/Hold
>> |
>> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
>> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
>> ||/ Name                               Version
>> Architecture           Description
>>
>> +++-==================================-======================-======================-=========================================================================
>> ii  lava-dispatcher                    2017.7-1~bpo9+1        amd64
>>               Linaro Automated Validation Architecture dispatcher
>> ii  lava-server                        2017.7-1~bpo9+1        all
>>               Linaro Automated Validation Architecture server
>> root@stretch:/etc/lava-server/dispatcher-config/devices#
>>
>> How I can upgrade to latest Lava. I know that apt-get upgrade Lava
>> does do the magic.
>
>
> The documentation covers upgrades -
> https://validation.linaro.org/static/docs/v2/installing_on_debian.html#lava-repositories
>
>>
>>
>> > Check the lava-master and lava-slave logs to find the misconfiguration.
>> > It's likely to be invalid state of the device and/or test job or
>> > misconfigured device configuration.
>>
>> No idea where the logs are? /var/log/apache2/lava-server.log ???
>
>
> This is also in the documentation.
> https://validation.linaro.org/static/docs/v2/simple-admin.html#where-to-find-debug-information
>
>>
>>
>> No idea what the output means.
>
>
> There is apache documentation to help you with this but apache is doing the
> serving of the pages, not the scheduling.
>
>>
>>
>> Thank you,
>> Zoran
>> _______
>>
>> On Wed, Feb 21, 2018 at 2:45 PM, Neil Williams <neil.williams@linaro.org>
>> wrote:
>> > On 21 February 2018 at 13:33, Zoran S <zoran.stojsavljevic.de@gmail.com>
>> > wrote:
>> >>
>> >> Hello to LAVA admins,
>> >>
>> >> You have fatal error in Lava V2. I submitted job, which hangs in
>> >> submitted state indefinitely.
>> >
>> >
>> > This will be down to local misconfiguration. Submitted jobs will stay in
>> > submitted until everything is ready for the job to start.
>> >
>> > What version of LAVA are you running? There were important changes in
>> > scheduling after the removal of V1 in 2018.1
>> >
>> >>
>> >> I also see the non-existent job 92 (which is deleted) running there?!
>> >
>> >
>> > Test jobs should not typically be deleted - except possibly as part of
>> > archival.
>> >
>> >>
>> >> http://localhost:8080/scheduler/device/bbb01
>> >>
>> >> 254 2018-02-21 12:05 Reserved → Running —
>> >> Job 92 running
>> >
>> >
>> > We don't use this format for device transitions anymore. There were
>> > issues
>> > with the old transitions but those could not be addressed until V1 was
>> > removed.
>> >
>> > https://validation.linaro.org/scheduler/device/beaglebone-black02
>> >
>> >>
>> >> 253 2018-02-21 12:04 Idle → Reserved —
>> >> Reserved for job 92
>> >> 252 2018-02-21 12:03 Running → Idle —
>> >> Job 91 cancelled
>> >> 251 2018-02-21 12:03 Reserved → Running —
>> >> Job 91 running
>> >> 250 2018-02-21 12:03 Idle → Reserved —
>> >> Reserved for job 91
>> >> 249 2018-02-21 11:33 Running → Idle —
>> >> Job 89 has ended. Setting job status Incomplete
>> >>
>> >> But when I look to this local pointer:
>> >> http://localhost:8080/admin/lava_scheduler_app/testjob/
>> >>
>> >> 102 Submittednobodylessbeaglebone-black -Feb. 21, 2018, 1:07 p.m.--
>> >> 76   Incompletenobodylessbeaglebone-black bbb01 (Running, health
>> >> Unknown)Feb. 20, 2018, 9:50 a.m.Feb. 20, 2018, 9:50 a.m.Feb. 20, 2018,
>> >> 9:55 a.m.
>> >> 49   Completenobodylessqemu qemu01 (Idle, health Unknown)Feb. 16,
>> >> 2018, 10:44 a.m.Feb. 16, 2018, 10:44 a.m.Feb. 16, 2018, 10:50 a.m.
>> >> 43   Completenobodylessqemu qemu01 (Idle, health Unknown)Feb. 16,
>> >> 2018, 8:42 a.m.Feb. 16, 2018, 8:43 a.m.Feb. 16, 2018, 8:49 a.m.
>> >>
>> >> How to force submitted job to be active job (from both CLI and GUI)?
>> >
>> >
>> > Check the lava-master and lava-slave logs to find the misconfiguration.
>> > It's
>> > likely to be invalid state of the device and/or test job or
>> > misconfigured
>> > device configuration.
>> >
>> >>
>> >>
>> >> How to delete nonexistent jobs (from both CLI and GUI)?
>> >
>> >
>> > Not recommended. lava-server manage has helpers to delete objects but
>> > the
>> > first thing to do is debug what is happening on your localhost.
>> >
>> >>
>> >>
>> >> Thank you,
>> >> Zoran
>> >> _______________________________________________
>> >> Lava-users mailing list
>> >> Lava-users@lists.linaro.org
>> >> https://lists.linaro.org/mailman/listinfo/lava-users
>> >
>> >
>> >
>> >
>> > --
>> >
>> > Neil Williams
>> > =============
>> > neil.williams@linaro.org
>> > http://www.linux.codehelp.co.uk/
>
>
>
>
> --
>
> Neil Williams
> =============
> neil.williams@linaro.org
> http://www.linux.codehelp.co.uk/