Hello,
I recently went "on a spree" to submit a number of LAVA UI issues either accumulated in my mind or just caught on exhaustive search for a feature I miss. They are at https://git.lavasoftware.org/lava/lava/-/issues?scope=all&author_usernam... and I believe all of them worth fixing (and don't bring in concerns like backward compatibility). So, they're up for triaging by the maintainers and feedback from the community to confirm they're indeed such. (Then, I'd be able to help with addressing them, as much as I can).
There're more issues, about which I'm less sure. So, instead of submitting such as bugs, let me ask for a feedback in the email, starting with this one:
When you prepare a job definition YAML to submit to LAVA, in it you something like:
job_name: 'zephyr-net-ping #823-2ac65a24'
It may be not immediately obvious, but what you had as a "job name" in the YAML, ends up as a "description" when you visit job list in the LAVA UI (E.g. at https://validation.linaro.org/scheduler/alljobs).
"Is it that much of a difference? Most people wouldn't even notice!" you may think, and I can only second that. I for one never paid attention to that for on-and-off, mostly very light usage of LAVA for some 10 years (i.e. I started yet with the previous version which might look different at all).
And indeed, it won't make any difference until may want to start querying job results. And I would like to share a story of what that difference costed me.
Here's a patch I submitted in June 2017: https://git.linaro.org/ci/job/configs.git/commit/lite-aeolus/lava-job-defini... with a commit message and inline comment of: "For some reason, LAVA doesn't allow to query by real job name, so we need to duplicate it as metadata."
Here's another patch which I submitted based on the analysis I present here: https://review.linaro.org/c/ci/job/configs/+/34677
So, for almost 3 years I lived with a misconception that LAVA can't query by a job name.
Call it an extreme case. Call me dumb. Say that there's autocompletion for field names and any person with some agility of mind would type different characters and would notice "description" as available field. Yeah, I did all that. But I'm with me from 3 years ago: for me, "job name" is "zephyr-net-ping #823-2ac65a24". "job description" would be something like "Ping Zephyr dumb_http_server sample with packets of different size and interval". To the best of my knowledge, there's no such a field in jobdef YAML to include such a description, and I definitely don't include it in my jobs, so never would I try to search anything in "description", for the lack thereof in my jobs.
Unless I have too much time to apply random "genetic algorithms" to LAVA UI or just do exhaustive search thru it. Which I usually don't have time for (I'm not a test engineer, but a developer, with "test that code" always in backlog). Until recently, when testing debt has really become pressing, and I decided to face all "skeletons in closed" I had with LAVA ;-).
So, even 3 years haven't passed until I "did my homework" on searching jobs by name. But my point would be that there should be no need for any "homework" regarding these matters, it all should just work right away without double-thinking and guesswork.
And I would consider this a genuine bug. I'm less sure about fixing though. Because it's not a matter of just changing a few UI labels. It's more pervasive, as e.g. this URL shows:
https://lite.validation.linaro.org/results/query/+custom?entity=testjob&...
And seeing that URL, one can actually conjecture how that "description" appeared in the UI: it's actually name of the database table. Which just bled into system public interface. Too bad, because another public interface uses "job name" for that field, and internal naming like db fields shouldn't prevail.
Anyway, I'm not going to argue that DB should be migrated, and it won't help, as we definitely don't want to break existing queries, etc. So, the only way to fix that would be to alias existing "description" to clearer "name" (or "jobname", "job_name"), and perform mapping on DB access. I'm note sure if it's worth. Based on my experience, I'd go for it, and it doesn't sounds like rocker science. But only maintainers could imagine how kludgy it would be in the actual LAVA database and if it's worth it, taking into account other tasks and priorities.
Thanks for reading the story (rant?).
On Tue, 31 Mar 2020 at 17:37, Paul Sokolovsky paul.sokolovsky@linaro.org wrote:
Hello,
I recently went "on a spree" to submit a number of LAVA UI issues either accumulated in my mind or just caught on exhaustive search for a feature I miss. They are at https://git.lavasoftware.org/lava/lava/-/issues?scope=all&author_usernam... and I believe all of them worth fixing (and don't bring in concerns like backward compatibility). So, they're up for triaging by the maintainers and feedback from the community to confirm they're indeed such. (Then, I'd be able to help with addressing them, as much as I can).
There're more issues, about which I'm less sure. So, instead of submitting such as bugs, let me ask for a feedback in the email, starting with this one:
When you prepare a job definition YAML to submit to LAVA, in it you something like:
job_name: 'zephyr-net-ping #823-2ac65a24'
It may be not immediately obvious, but what you had as a "job name" in the YAML, ends up as a "description" when you visit job list in the LAVA UI (E.g. at https://validation.linaro.org/scheduler/alljobs).
"Is it that much of a difference? Most people wouldn't even notice!" you may think, and I can only second that. I for one never paid attention to that for on-and-off, mostly very light usage of LAVA for some 10 years (i.e. I started yet with the previous version which might look different at all).
And indeed, it won't make any difference until may want to start querying job results. And I would like to share a story of what that difference costed me.
Here's a patch I submitted in June 2017: https://git.linaro.org/ci/job/configs.git/commit/lite-aeolus/lava-job-defini... with a commit message and inline comment of: "For some reason, LAVA doesn't allow to query by real job name, so we need to duplicate it as metadata."
Here's another patch which I submitted based on the analysis I present here: https://review.linaro.org/c/ci/job/configs/+/34677
So, for almost 3 years I lived with a misconception that LAVA can't query by a job name.
Call it an extreme case. Call me dumb. Say that there's autocompletion for field names and any person with some agility of mind would type different characters and would notice "description" as available field. Yeah, I did all that. But I'm with me from 3 years ago: for me, "job name" is "zephyr-net-ping #823-2ac65a24". "job description" would be something like "Ping Zephyr dumb_http_server sample with packets of different size and interval". To the best of my knowledge, there's no such a field in jobdef YAML to include such a description, and I definitely don't include it in my jobs, so never would I try to search anything in "description", for the lack thereof in my jobs.
Unless I have too much time to apply random "genetic algorithms" to LAVA UI or just do exhaustive search thru it. Which I usually don't have time for (I'm not a test engineer, but a developer, with "test that code" always in backlog). Until recently, when testing debt has really become pressing, and I decided to face all "skeletons in closed" I had with LAVA ;-).
So, even 3 years haven't passed until I "did my homework" on searching jobs by name. But my point would be that there should be no need for any "homework" regarding these matters, it all should just work right away without double-thinking and guesswork.
And I would consider this a genuine bug. I'm less sure about fixing though. Because it's not a matter of just changing a few UI labels. It's more pervasive, as e.g. this URL shows:
https://lite.validation.linaro.org/results/query/+custom?entity=testjob&...
And seeing that URL, one can actually conjecture how that "description" appeared in the UI: it's actually name of the database table. Which just bled into system public interface. Too bad, because another public interface uses "job name" for that field, and internal naming like db fields shouldn't prevail.
This is confirmed theory, not just conjecture :) https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode... https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode...
Anyway, I'm not going to argue that DB should be migrated, and it won't help, as we definitely don't want to break existing queries, etc. So, the only way to fix that would be to alias existing "description" to clearer "name" (or "jobname", "job_name"), and perform mapping on DB access. I'm note sure if it's worth. Based on my experience, I'd go for it, and it doesn't sounds like rocker science. But only maintainers could imagine how kludgy it would be in the actual LAVA database and if it's worth it, taking into account other tasks and priorities.
Maybe the simplest solution would be changing this field in YAML and calling it 'description' instead of 'name'? For some time (transition period) both ways should be possible but in the long run we would stop using 'name'.
milosz
Thanks for reading the story (rant?).
-- Best Regards, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
Hello Milosz,
On Wed, 1 Apr 2020 11:01:17 +0000 Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
[]
job_name: 'zephyr-net-ping #823-2ac65a24'
It may be not immediately obvious, but what you had as a "job name" in the YAML, ends up as a "description" when you visit job list in the LAVA UI (E.g. at https://validation.linaro.org/scheduler/alljobs).
https://lite.validation.linaro.org/results/query/+custom?entity=testjob&...
And seeing that URL, one can actually conjecture how that "description" appeared in the UI: it's actually name of the database table. Which just bled into system public interface. Too bad, because another public interface uses "job name" for that field, and internal naming like db fields shouldn't prevail.
This is confirmed theory, not just conjecture :) https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode... https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode...
Thanks for confirming that.
Anyway, I'm not going to argue that DB should be migrated, and it won't help, as we definitely don't want to break existing queries, etc. So, the only way to fix that would be to alias existing "description" to clearer "name" (or "jobname", "job_name"), and perform mapping on DB access. I'm note sure if it's worth. Based on my experience, I'd go for it, and it doesn't sounds like rocker science. But only maintainers could imagine how kludgy it would be in the actual LAVA database and if it's worth it, taking into account other tasks and priorities.
Maybe the simplest solution would be changing this field in YAML and calling it 'description' instead of 'name'? For some time (transition period) both ways should be possible but in the long run we would stop using 'name'.
Hmm, at first look, this seems like "upside down" solution. A model of LAVA I would have is that the "primary external interface" of LAVA is job/test definitions and ways to submit them (APIs/tools). The rest is "merely" a "UI frontend". Saying, after so many years, that UI frontend has got it right, while job specification format was wrong all this time is ... strange. Especially if we border on saying "DB has got it right", with DB field names being a hidden implementation detail.
Migration cost would definitely be high too, with many jobdefs in the wild, which would need to be updated (or if they don't, confusion persists). In my plan, no active updates on users' side is needed.
Also as I mentioned in the original email, for me typical examples of name vs description:
name: zephyr-net-ping #823-2ac65a24 description: Ping Zephyr dumb_http_server sample with packets of different size and interval.
That said, I don't know. If YAML's field was called "description" in the first place, we all would take for granted that it is "short (not too long) sequence of characters", and there certainly wouldn't be the confusion described.
So, in the end, the relatively high migration cost (to the users) is the main concern.
milosz
Thanks, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
Hello Paul,
thanks for your mail.
As said by Milosz, the job_name is saved into the TestJob.description field.
I agree that the name has been badly chosen but fixing this will generate a lot of work for no real benefits.
We can: * rename the database field. In this case we will have to change a lot of places in the source code and in the API return values. * rename job_name to description in the job definition. In this case, that's mainly some work for users.
So in both situations users will have to update either job definition or api scripts.
So I believe that this is way too much work for no real benefit.
Thanks
Le mer. 1 avr. 2020 à 14:50, Paul Sokolovsky paul.sokolovsky@linaro.org a écrit :
Hello Milosz,
On Wed, 1 Apr 2020 11:01:17 +0000 Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
[]
job_name: 'zephyr-net-ping #823-2ac65a24'
It may be not immediately obvious, but what you had as a "job name" in the YAML, ends up as a "description" when you visit job list in the LAVA UI (E.g. at https://validation.linaro.org/scheduler/alljobs).
https://lite.validation.linaro.org/results/query/+custom?entity=testjob&...
And seeing that URL, one can actually conjecture how that "description" appeared in the UI: it's actually name of the database table. Which just bled into system public interface. Too bad, because another public interface uses "job name" for that field, and internal naming like db fields shouldn't prevail.
This is confirmed theory, not just conjecture :)
https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode...
https://git.lavasoftware.org/lava/lava/-/blob/master/lava_scheduler_app/mode...
Thanks for confirming that.
Anyway, I'm not going to argue that DB should be migrated, and it won't help, as we definitely don't want to break existing queries, etc. So, the only way to fix that would be to alias existing "description" to clearer "name" (or "jobname", "job_name"), and perform mapping on DB access. I'm note sure if it's worth. Based on my experience, I'd go for it, and it doesn't sounds like rocker science. But only maintainers could imagine how kludgy it would be in the actual LAVA database and if it's worth it, taking into account other tasks and priorities.
Maybe the simplest solution would be changing this field in YAML and calling it 'description' instead of 'name'? For some time (transition period) both ways should be possible but in the long run we would stop using 'name'.
Hmm, at first look, this seems like "upside down" solution. A model of LAVA I would have is that the "primary external interface" of LAVA is job/test definitions and ways to submit them (APIs/tools). The rest is "merely" a "UI frontend". Saying, after so many years, that UI frontend has got it right, while job specification format was wrong all this time is ... strange. Especially if we border on saying "DB has got it right", with DB field names being a hidden implementation detail.
Migration cost would definitely be high too, with many jobdefs in the wild, which would need to be updated (or if they don't, confusion persists). In my plan, no active updates on users' side is needed.
Also as I mentioned in the original email, for me typical examples of name vs description:
name: zephyr-net-ping #823-2ac65a24 description: Ping Zephyr dumb_http_server sample with packets of different size and interval.
That said, I don't know. If YAML's field was called "description" in the first place, we all would take for granted that it is "short (not too long) sequence of characters", and there certainly wouldn't be the confusion described.
So, in the end, the relatively high migration cost (to the users) is the main concern.
milosz
Thanks, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users
lava-users@lists.lavasoftware.org
-
Milosz Wasilewski -
Paul Sokolovsky -
Remi Duraffort