On Wed, 8 Apr 2020 at 16:44, Paul Sokolovsky paul.sokolovsky@linaro.org wrote:
On Wed, 8 Apr 2020 11:43:38 +0000 Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
[]
https://lite.validation.linaro.org/results/query/+custom?entity=testjob&...
(Our jobs are submit-proxied via SQUAD from automated Jenkins builds.)
Hmm, could you explain (maybe again) why are you doing that while the information you require is already aggregated. Example: https://qa-reports.linaro.org/lite/zephyr-upstream/build/295572a5
Milosz, thanks for insightful discussion we had at the LAVA Design online meeting, and an offer to guide me thru SQUAD on how to address my usecases. As you suggest, let's move that to private mail to not offtopic on this list.
But just to not leave this thread to end abruptly in the archive, let me tell what's the problem I see immediately at the link you gave: the page starts with:
Test jobs 560 61
Moving cursor at "560", I see "Complete", at "61" - "Incomplete". And as I explained, "Complete" number is rather useless to me, because some "Complete" jobs are actually "failed" jobs.
I would say this is only informational as when you're using SQUAD test jobs become irrelevant.
It occurred to me that maybe it's unclear what I'm talking about, because you may picture "LITE jobs" as instance of zephyr-upstream Jenkins build, while I'm trying to think of how to setup new jobs, so they weren't as deeply problematic as zephyr-upstream is. So, example of "new" job is https://lite.validation.linaro.org/scheduler/job/954523 - "Complete" and having 1 failed testcase, so "failed" job overall, per my criteria.
My assumption might be wrong, but I think you're looking for failed tests, rather than 'failed jobs'. This is the message I'm trying to convey all the time. LAVA is an executor and LAVA jobs are means to execute your tests. Job status isn't really that useful when you try to check which tests are passed and which are failed. For example there can be incomplete job with useful test results and complete job without any test results. How would you classify this?
That job didn't go thru SQUAD, so I guess my next step is doing homework with passing that job thru SQUAD, to see if I have any issues with how its results will be presented there.
Sure, if you need any help, just ask.
milosz
milosz
[]
-- Best Regards, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog