On 17/01/2018 11:28, Neil Williams wrote:
On 17 January 2018 at 10:04, Loys Ollivier lollivier@baylibre.com wrote:
On 17/01/2018 10:52, Neil Williams wrote:
On 17 January 2018 at 09:48, Loys Ollivier lollivier@baylibre.com
wrote:
Hi Stevan, all,
Following on that here's the output of the lava-tool using version 0.24-1. Seems like there's a new bug:
--8>
$ lava-tool wait-job-events http://loys@lava-test:10080
--job-definition
job.yaml submitted as job(s): http://lava-test:10080/scheduler/job/7528
This isn't a typical way to run LAVA, there's no guarantee that it will work on a random port - make sure you're not trying to run LAVA as if it was just another django project, it is not. The django internal support
is
not supported for LAVA. It needs to be packaged (using the developer
build
scripts if you want to build latest) and have apache properly configured. (It's also advisable to use https:// to protect the authentication
tokens.)
Thank you for your feedback. Although this is not a typical way, I am using it on a local network and it has been working properly for several months.
There is no guarantee that it will continue to work and bugs or issues in that setup *must* be replicated on a supported system first.
As it is my testing instance (to not break production) the tokens are not sensitive and anyway the network is not connected to the outside world.
That answers for the https:// but there is still a problem with the test instance setup using a random port. Make sure it is installed from packages and is the latest release, 2018.1.
Are you trying to run the test instance from the django testproject support?
I am running a docker container which redirects port 10080 -> 80. I don't think it changes the lava behavior within the container.
Regarding the latest bug mentioned, it seems like the key name has changed in the python dictionary. Playing around with packaging, random port number or auth tokens won't help.
There is a code change in the scheduler - your test instance must be on 2018.1 for the call in lava-tool 0.24 to work.
The code as-is does work with those in combination:
$ lava-tool wait-job-events --job-definition ~/code/lava/pipeline/git/lxc-ubuntu.yaml local submitted as job(s): http://localhost/scheduler/job/12502 Now waiting for job events... {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'job': ' http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Scheduled', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Running', 'health': 'Unknown', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'} {'pipeline': True, 'description': 'LXC Ubuntu Xenial', 'start_time': '2018-01-17T10:25:09.425483+00:00', 'job': 'http://localhost/scheduler/12502', 'visibility': 'Publicly visible', 'priority': 50, 'device': 'lxc-01', 'state': 'Finished', 'health': 'Incomplete', 'end_time': '2018-01-17T10:25:12.979031+00:00', 'device_type': 'lxc', 'submit_time': '2018-01-17T10:25:03.621066+00:00', 'submitter': 'neil.williams'}
The problem is with the data structure returned by your test instance, it's not a problem in lava-tool which has been updated for those changes.
$ dpkg -l lava-server lava-dispatcher Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=============================-===================-===================-================================================================ ii lava-dispatcher 2018.1+4769.7a93650 amd64 Linaro Automated Validation Architecture dispatcher ii lava-server 2018.1+7019.756d2e6 all Linaro Automated Validation Architecture server
Ok, thank you for the pointer. I wasn't aware that lava-tool version requires a certain version of lava-server. Maybe this could be checked in lava-tool to avoid this bug ? Like a message reporting that the lava-server version must be >= 2018.01 for instance ? I will try to upgrade lava and test with the new instance.
Now waiting for job events... Traceback (most recent call last): File "/usr/bin/lava-tool", line 11, in <module> load_entry_point('lava-tool==0.24', 'console_scripts',
'lava-tool')()
File "/usr/lib/python2.7/dist-packages/lava_tool/dispatcher.py", line 49, in main LavaDispatcher.run()
File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line
150, in run raise SystemExit(cls().dispatch(args)) File "/usr/lib/python2.7/dist-packages/lava/tool/dispatcher.py", line 140, in dispatch return command.invoke() File "/usr/lib/python2.7/dist-packages/lava_scheduler_tool/commands.py",
line
820, in invoke if data["state"] == self.FINISHED_JOB_STATE: KeyError: 'state'
Make sure the LAVA instance itself is running 2018.1 as there have been changes to the scheduler state machine.
On 16/12/2017 10:28, Stevan Radakovic wrote:
Hi Loys,
After some more testing I realized it's a bug in lava-tool which causes a problem when you're using a specific port for the lava-server endpoint. It's a pretty straightforward fix, here's the tracking ticket for it: https://projects.linaro.org/servicedesk/customer/portal/1/
CTT-819
Thanks,
On 12/14/2017 05:56 PM, Loys Ollivier wrote:
On 14/12/2017 16:18, Stevan Radakovic wrote: > Hi Loys, > > There's auth-add command which is used to add credentials for > authenticatation of lava-tool to the lava-server instance. But if you > can submit a job via lava-tool you've probably already done that. > > Are you receiving no output after submitting the job via lava-tool > wait-job-events? > Exactly. > It's possibly a bug in lava-tool I'm also seeing job completed event > missing when trying it out locally. I'll investigate some more and
file
> a bug. > > lava-tool exits when it receives job completed event, that's the most > likely reason that it hangs. Not sure to get your point here. But I think we're on the same page: lava-tool hangs...
Thanks for help !
Is there any way I can follow the evolution / resolution of this bug ?
> Cheers, > > On 12/14/2017 03:35 PM, Loys Ollivier wrote: >> Hello Stevan, >> >> Thanks for the help on that. I enabled the server side
notifications,
>> I can see them using a publisher + python on my host computer. >> Unfortunately lava-tool still doesn't return even when the job >> finishes.... >> >> Also, my server is running on a specific port so to access it I am >> using: lava-tool wait-job-events http://user@server:port/RPC2 >> It works perfectly well to submit jobs. I also checked that the port >> number 5500 is opened. >> >> Is there anything else I should do on the lava-tool side to get this >> notifications ? Again I can receive them on the host using python
but
>> for some reason lava-tool doesn't work. >> >> Thanks ! >> >> Loys >> >> On Wed, Dec 13, 2017 at 4:47 PM, Stevan Radakovic >> <stevan.radakovic@linaro.org mailto:stevan.radakovic@linaro.org> >> wrote: >> >> Hi Loys, >> >> There's definitely bug with the message you get from the >> "submit-job --block" and we will open a ticket for this - >> wait-for-job does not exist it was replaced with
wait-job-events
>> long time ago. >> >> For using it, you need to enable event notification on your
server
>> first. Please follow these instructions: >> >> >> https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications
>> >> >> >> <https://staging.validation.linaro.org/static/docs/v2/
advanced-installation.html#configuring-event-notifications>
>> >> >> >> After that, using lava-tool, you can either submit and wait for >> the job events or track the events of the existing job. Example >> you provided is good for the latter example. >> >> To submit and wait you can use: >> lava-tool wait-job-events --job-definition /path/to/job.yaml >> http://your.server/RPC2/ >> >> In either case, lava-tool should exit when it receives job >> 'completed' event from the server. >> >> >> HTH, >> >> >> >> On 12/13/2017 02:54 PM, Loys Ollivier wrote: >>> Hello everyone, >>> >>> I am using lava-tool to monitor my jobs. Previously I used: >>> $ lava-tool submit-job --block >>> >>> Using version of lava-tool 0.23 I now have this message: >>> --> This kind of polling is deprecated and will be removed in
the
>>> next release. Please use "wait-for-job" command. >>> >>> But "wait-for-job" doesn't exist. >>> There is a "wait-job-events" option though. I tried this one
and
>>> it doesn't return even once the job has finished. If I
manually
>>> stop it and restart it with the same job number I get as
output:
>>> --> Job already finished with status Complete. >>> >>> Command I'm using: >>> $ lava-tool wait-job-events --job-id 20
>>> >>> Is there anything I'm doing incorrectly ? Or are you aware of >>> this bug ? >>> >>> Thanks ! >>> >>> >>> -- Loys OLLIVIER >>> >>> >>> >>> >>> _______________________________________________ >>> Lava-users mailing list >>> Lava-users@lists.linaro.org <mailto:Lava-users@lists.
linaro.org
>>> https://lists.linaro.org/mailman/listinfo/lava-users >>> https://lists.linaro.org/mailman/listinfo/lava-users >> -- Stevan Radaković | LAVA Engineer >> Linaro.org <www.linaro.org http://www.linaro.org> │ Open
source
>> software for ARM SoCs >> >> >> >> >> -- >> Loys OLLIVIER >> Baylibre >>
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users