On 17/01/2018 11:28, Neil Williams wrote:
> On 17 January 2018 at 10:04, Loys Ollivier <lollivier@baylibre.com> wrote:
>
>>
>>
>> On 17/01/2018 10:52, Neil Williams wrote:
>>> On 17 January 2018 at 09:48, Loys Ollivier <lollivier@baylibre.com>
>> wrote:
>>>
>>>>
>>>> Hi Stevan, all,
>>>>
>>>> Following on that here's the output of the lava-tool using version
>>>> 0.24-1. Seems like there's a new bug:
>>>>
>>>> --8>
>>>>
>>>> $ lava-tool wait-job-events http://loys@lava-test:10080
>> --job-definition
>>>> job.yaml
>>>> submitted as job(s):
>>>> http://lava-test:10080/scheduler/job/7528
>>>
>>>
>>> This isn't a typical way to run LAVA, there's no guarantee that it will
>>> work on a random port - make sure you're not trying to run LAVA as if it
>>> was just another django project, it is not. The django internal support
>> is
>>> not supported for LAVA. It needs to be packaged (using the developer
>> build
>>> scripts if you want to build latest) and have apache properly configured.
>>> (It's also advisable to use https:// to protect the authentication
>> tokens.)
>>>
>>>
>>
>> Thank you for your feedback. Although this is not a typical way, I am
>> using it on a local network and it has been working properly for several
>> months.
>
>
> There is no guarantee that it will continue to work and bugs or issues in
> that setup *must* be replicated on a supported system first.
>
>
>> As it is my testing instance (to not break production) the
>> tokens are not sensitive and anyway the network is not connected to the
>> outside world.
>>
>
> That answers for the https:// but there is still a problem with the test
> instance setup using a random port. Make sure it is installed from packages
> and is the latest release, 2018.1.
>
> Are you trying to run the test instance from the django testproject support?
>
>

I am running a docker container which redirects port 10080 -> 80. I
don't think it changes the lava behavior within the container.

>> Regarding the latest bug mentioned, it seems like the key name has
>> changed in the python dictionary. Playing around with packaging, random
>> port number or auth tokens won't help.
>>
>>
> There is a code change in the scheduler - your test instance must be on
> 2018.1 for the call in lava-tool 0.24 to work.
>
> The code as-is does work with those in combination:
>
> $ lava-tool wait-job-events --job-definition
> ~/code/lava/pipeline/git/lxc-ubuntu.yaml local
> submitted as job(s):
> http://localhost/scheduler/job/12502
> Now waiting for job events...
> {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': '
> http://localhost/scheduler/12502', 'visibility': 'Publicly visible',
> 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health':
> 'Unknown', 'device_type': 'lxc', 'submit_time':
> '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
> {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time':
> '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502',
> 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01',
> 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc',
> 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter':
> 'neil.williams'}
> {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time':
> '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502',
> 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01',
> 'state': 'Finished', 'health': 'Incomplete', 'end_time':
> '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time':
> '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
>
> The problem is with the data structure returned by your test instance, it's
> not a problem in lava-tool which has been updated for those changes.
>
> $ dpkg -l lava-server lava-dispatcher
> Desired=Unknown/Install/Remove/Purge/Hold
> |
> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
> ||/ Name                          Version             Architecture
> Description
> +++-=============================-===================-===================-================================================================
> ii  lava-dispatcher               2018.1+4769.7a93650 amd64
>  Linaro Automated Validation Architecture dispatcher
> ii  lava-server                   2018.1+7019.756d2e6 all
>  Linaro Automated Validation Architecture server
>
>
>
>

Ok, thank you for the pointer. I wasn't aware that lava-tool version
requires a certain version of lava-server. Maybe this could be checked
in lava-tool to avoid this bug ? Like a message reporting that the
lava-server version must be >= 2018.01 for instance ?

I will try to upgrade lava and test with the new instance.

>>>
>>>> Now waiting for job events...
>>>> Traceback (most recent call last):
>>>>   File "/usr/bin/lava-tool", line 11, in <module>
>>>>     load_entry_point('lava-tool==0.24', 'console_scripts',
>> 'lava-tool')()
>>>>   File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line
>>>> 49, in main
>>>>     LavaDispatcher.run()
>>
>>>   File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line
>>>> 150, in run
>>>>     raise SystemExit(cls().dispatch(args))
>>>>   File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line
>>>> 140, in dispatch
>>>>     return command.invoke()
>>>>   File
>>>> "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
>> line
>>>> 820, in invoke
>>>>     if data["state"] == self.FINISHED_JOB_STATE:
>>>> KeyError: 'state'
>>>>
>>>
>>> Make sure the LAVA instance itself is running 2018.1 as there have been
>>> changes to the scheduler state machine.
>>>
>>>
>>>
>>>>
>>>>
>>>> On 16/12/2017 10:28, Stevan Radakovic wrote:
>>>>> Hi Loys,
>>>>>
>>>>> After some more testing I realized it's a bug in lava-tool which causes
>>>>> a problem when you're using a specific port for the lava-server
>>>>> endpoint. It's a pretty straightforward fix, here's the tracking ticket
>>>>> for it: https://projects.linaro.org/servicedesk/customer/portal/1/
>>>> CTT-819
>>>>>
>>>>> Thanks,
>>>>>
>>>>> On 12/14/2017 05:56 PM, Loys Ollivier wrote:
>>>>>>
>>>>>> On 14/12/2017 16:18, Stevan Radakovic wrote:
>>>>>>> Hi Loys,
>>>>>>>
>>>>>>> There's auth-add command which is used to add credentials for
>>>>>>> authenticatation of lava-tool to the lava-server instance. But if you
>>>>>>> can submit a job via lava-tool you've probably already done that.
>>>>>>>
>>>>>>> Are you receiving no output after submitting the job via lava-tool
>>>>>>> wait-job-events?
>>>>>>>
>>>>>> Exactly.
>>>>>>> It's possibly a bug in lava-tool I'm also seeing job completed event
>>>>>>> missing when trying it out locally. I'll investigate some more and
>> file
>>>>>>> a bug.
>>>>>>>
>>>>>>> lava-tool exits when it receives job completed event, that's the most
>>>>>>> likely reason that it hangs.
>>>>>> Not sure to get your point here. But I think we're on the same page:
>>>>>> lava-tool hangs...
>>>>>>
>>>>>> Thanks for help !
>>>>>>
>>>>>> Is there any way I can follow the evolution / resolution of this bug ?
>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> On 12/14/2017 03:35 PM, Loys Ollivier wrote:
>>>>>>>> Hello Stevan,
>>>>>>>>
>>>>>>>> Thanks for the help on that. I enabled the server side
>> notifications,
>>>>>>>> I can see them using a publisher + python on my host computer.
>>>>>>>> Unfortunately lava-tool still doesn't return even when the job
>>>>>>>> finishes....
>>>>>>>>
>>>>>>>> Also, my server is running on a specific port so to access it I am
>>>>>>>> using: lava-tool wait-job-events http://user@server:port/RPC2
>>>>>>>> It works perfectly well to submit jobs. I also checked that the port
>>>>>>>> number 5500 is opened.
>>>>>>>>
>>>>>>>> Is there anything else I should do on the lava-tool side to get this
>>>>>>>> notifications ? Again I can receive them on the host using python
>> but
>>>>>>>> for some reason lava-tool doesn't work.
>>>>>>>>
>>>>>>>> Thanks !
>>>>>>>>
>>>>>>>> Loys
>>>>>>>>
>>>>>>>> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic
>>>>>>>> <stevan.radakovic@linaro.org <mailto:stevan.radakovic@linaro.org>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>      Hi Loys,
>>>>>>>>
>>>>>>>>      There's definitely bug with the message you get from the
>>>>>>>>      "submit-job --block" and we will open a ticket for this -
>>>>>>>>      wait-for-job does not exist it was replaced with
>> wait-job-events
>>>>>>>>      long time ago.
>>>>>>>>
>>>>>>>>      For using it, you need to enable event notification on your
>>>> server
>>>>>>>>      first. Please follow these instructions:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://staging.validation.linaro.org/static/docs/v2/
>>>> advanced-installation.html#configuring-event-notifications
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> <https://staging.validation.linaro.org/static/docs/v2/
>>>> advanced-installation.html#configuring-event-notifications>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>      After that, using lava-tool, you can either submit and wait for
>>>>>>>>      the job events or track the events of the existing job. Example
>>>>>>>>      you provided is good for the latter example.
>>>>>>>>
>>>>>>>>      To submit and wait you can use:
>>>>>>>>      lava-tool wait-job-events --job-definition /path/to/job.yaml
>>>>>>>>      http://your.server/RPC2/
>>>>>>>>
>>>>>>>>      In either case, lava-tool should exit when it receives job
>>>>>>>>      'completed' event from the server.
>>>>>>>>
>>>>>>>>
>>>>>>>>      HTH,
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>      On 12/13/2017 02:54 PM, Loys Ollivier wrote:
>>>>>>>>>      Hello everyone,
>>>>>>>>>
>>>>>>>>>      I am using lava-tool to monitor my jobs. Previously I used:
>>>>>>>>>      $ lava-tool submit-job --block
>>>>>>>>>
>>>>>>>>>      Using version of lava-tool 0.23 I now have this message:
>>>>>>>>>      --> This kind of polling is deprecated and will be removed in
>>>> the
>>>>>>>>>      next release. Please use "wait-for-job" command.
>>>>>>>>>
>>>>>>>>>      But "wait-for-job" doesn't exist.
>>>>>>>>>      There is a "wait-job-events" option though. I tried this one
>> and
>>>>>>>>>      it doesn't return even once the job has finished. If I
>> manually
>>>>>>>>>      stop it and restart it with the same job number I get as
>> output:
>>>>>>>>>      --> Job already finished with status Complete.
>>>>>>>>>
>>>>>>>>>      Command I'm using:
>>>>>>>>>      $ lava-tool wait-job-events --job-id 20
>> http://user@lava-server
>>>>>>>>>
>>>>>>>>>      Is there anything I'm doing incorrectly ? Or are you aware of
>>>>>>>>>      this bug ?
>>>>>>>>>
>>>>>>>>>      Thanks !
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>      --     Loys OLLIVIER
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>      _______________________________________________
>>>>>>>>>      Lava-users mailing list
>>>>>>>>>      Lava-users@lists.linaro.org <mailto:Lava-users@lists.
>> linaro.org
>>>>>
>>>>>>>>>      https://lists.linaro.org/mailman/listinfo/lava-users
>>>>>>>>>      <https://lists.linaro.org/mailman/listinfo/lava-users>
>>>>>>>>      --     Stevan Radaković | LAVA Engineer
>>>>>>>>      Linaro.org <www.linaro.org <http://www.linaro.org>> │ Open
>>>> source
>>>>>>>> software for ARM SoCs
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Loys OLLIVIER
>>>>>>>> Baylibre
>>>>>>>>
>>>>>
>>>> _______________________________________________
>>>> Lava-users mailing list
>>>> Lava-users@lists.linaro.org
>>>> https://lists.linaro.org/mailman/listinfo/lava-users
>>>>
>>>
>>>
>>>
>>
>
>
>