On 17/01/2018 12:02, Neil Williams wrote:
On 17 January 2018 at 10:47, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com
wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if
it
was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly
configured.
(It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from
packages
and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject
support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Right, that makes sense. OK - it was a throwback to another issue where the test instance was poorly configured. The redirection happened to use the same kind of URL.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/
12502',
'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance,
it's
not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/
trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-======
=============-==============================================
ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ?
Normally, we would do that on the server side through the XML-RPC interface
- deprecate the old call and add a new one. lava-tool is beginning to show
problems from a lot of historical legacy code and it's not at all easy to remove as we did with the removal of V1. We're looking at a new tool with a clean Python3 start as a replacement. Once we have moved forward a bit more with that one, I'll make the announcement.
I understand thanks.
In the meantime, it's only this particular functionality in lava-tool which is affected. Are you using this particular support frequently? We can look at using the XML-RPC call system.api_version from lava-tool to check the particular instance being used for the wait-job-events support.
Here's what I am using in my CI loop: # lava-tool submit-job http://loys@lava-test:10080 job.yaml --block submitted as job: http://lava-test:10080/scheduler/job/75
This kind of polling is deprecated and will be removed in the next release. Please use "wait-for-job" command.
Hence my question about wait-job-events. As I did not want to use a deprecated command. I guess I will stick to that one until the new announcement. I do not have particular issues with the submit-job command.
I will try to upgrade lava and test with the new instance.
Thanks, sorry for any confusion - let us know the results either way.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py",
line
49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py",
line
140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote: > Hi Loys, > > After some more testing I realized it's a bug in lava-tool which
causes
> a problem when you're using a specific port for the lava-server > endpoint. It's a pretty straightforward fix, here's the tracking
ticket
> for it: https://projects.linaro.org/servicedesk/customer/portal/1/ CTT-819 > > Thanks, > > On 12/14/2017 05:56 PM, Loys Ollivier wrote: >> >> On 14/12/2017 16:18, Stevan Radakovic wrote: >>> Hi Loys, >>> >>> There's auth-add command which is used to add credentials for >>> authenticatation of lava-tool to the lava-server instance. But if
you
>>> can submit a job via lava-tool you've probably already done that. >>> >>> Are you receiving no output after submitting the job via lava-tool >>> wait-job-events? >>> >> Exactly. >>> It's possibly a bug in lava-tool I'm also seeing job completed
event
>>> missing when trying it out locally. I'll investigate some more and
file
>>> a bug. >>> >>> lava-tool exits when it receives job completed event, that's the
most
>>> likely reason that it hangs. >> Not sure to get your point here. But I think we're on the same page: >> lava-tool hangs... >> >> Thanks for help ! >> >> Is there any way I can follow the evolution / resolution of this
bug ?
>> >>> Cheers, >>> >>> On 12/14/2017 03:35 PM, Loys Ollivier wrote: >>>> Hello Stevan, >>>> >>>> Thanks for the help on that. I enabled the server side
notifications,
>>>> I can see them using a publisher + python on my host computer. >>>> Unfortunately lava-tool still doesn't return even when the job >>>> finishes.... >>>> >>>> Also, my server is running on a specific port so to access it I am >>>> using: lava-tool wait-job-events http://user@server:port/RPC2 >>>> It works perfectly well to submit jobs. I also checked that the
port
>>>> number 5500 is opened. >>>> >>>> Is there anything else I should do on the lava-tool side to get
this
>>>> notifications ? Again I can receive them on the host using python
but
>>>> for some reason lava-tool doesn't work. >>>> >>>> Thanks ! >>>> >>>> Loys >>>> >>>> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >>>> <stevan.radakovic@linaro.org <mailto:stevan.radakovic@linaro.org
>>>> wrote: >>>> >>>> Hi Loys, >>>> >>>> There's definitely bug with the message you get from the >>>> "submit-job --block" and we will open a ticket for this - >>>> wait-for-job does not exist it was replaced with
wait-job-events
>>>> long time ago. >>>> >>>> For using it, you need to enable event notification on your server >>>> first. Please follow these instructions: >>>> >>>> >>>> https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications >>>> >>>> >>>> >>>> https://staging.validation.linaro.org/static/docs/v2/ advanced-installation.html#configuring-event-notifications >>>> >>>> >>>> >>>> After that, using lava-tool, you can either submit and wait
for
>>>> the job events or track the events of the existing job.
Example
>>>> you provided is good for the latter example. >>>> >>>> To submit and wait you can use: >>>> lava-tool wait-job-events --job-definition /path/to/job.yaml >>>> http://your.server/RPC2/ >>>> >>>> In either case, lava-tool should exit when it receives job >>>> 'completed' event from the server. >>>> >>>> >>>> HTH, >>>> >>>> >>>> >>>> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>>>> Hello everyone, >>>>> >>>>> I am using lava-tool to monitor my jobs. Previously I used: >>>>> $ lava-tool submit-job --block >>>>> >>>>> Using version of lava-tool 0.23 I now have this message: >>>>> --> This kind of polling is deprecated and will be removed
in
the >>>>> next release. Please use "wait-for-job" command. >>>>> >>>>> But "wait-for-job" doesn't exist. >>>>> There is a "wait-job-events" option though. I tried this one
and
>>>>> it doesn't return even once the job has finished. If I
manually
>>>>> stop it and restart it with the same job number I get as
output:
>>>>> --> Job already finished with status Complete. >>>>> >>>>> Command I'm using: >>>>> $ lava-tool wait-job-events --job-id 20
>>>>> >>>>> Is there anything I'm doing incorrectly ? Or are you aware
of
>>>>> this bug ? >>>>> >>>>> Thanks ! >>>>> >>>>> >>>>> -- Loys OLLIVIER >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Lava-users mailing list >>>>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
> >>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>>> https://lists.linaro.org/mailman/listinfo/lava-users >>>> -- Stevan Radaković | LAVA Engineer >>>> Linaro.org <www.linaro.org http://www.linaro.org> │ Open source >>>> software for ARM SoCs >>>> >>>> >>>> >>>> >>>> -- >>>> Loys OLLIVIER >>>> Baylibre >>>> > _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users