Hi,
As stated in the subject, I was wondering if it is at all possible to use a repo on Azure DevOps instead of a GitHub one. I have been using a GitHub repo thus far, and it works just fine, but I need to move my files over to DevOps.
According to the docs here: https://docs.lavasoftware.org/lava/actions-test.html only git is supported at the moment, with URL and tar planned for. Would URL work for this if support came in at any point? Or would azure need its own support?
Or should git work and there is something wrong with the repo URL I am using? Because the particular error I get is that the password could not be read (I'm using a PAT in the link, like I did for GitHub)
Best Regards. Michael
Hi Michael,
I think the simple answer is that LAVA only supports git repositories without passwords. I think it might be possible to store the credentials in git global config on the dispatcher, but I haven't tried this. Under the hood the usual git client is used. If there is a way to configure credential manager for git not to ask any interactive questions, this might work.
Best Regards, Milosz
On Wed, Jan 24, 2024 at 10:34 PM michael.peddie@gallagher.com wrote:
Hi,
As stated in the subject, I was wondering if it is at all possible to use a repo on Azure DevOps instead of a GitHub one. I have been using a GitHub repo thus far, and it works just fine, but I need to move my files over to DevOps.
According to the docs here: https://docs.lavasoftware.org/lava/actions-test.html only git is supported at the moment, with URL and tar planned for. Would URL work for this if support came in at any point? Or would azure need its own support?
Or should git work and there is something wrong with the repo URL I am using? Because the particular error I get is that the password could not be read (I'm using a PAT in the link, like I did for GitHub)
Best Regards. Michael _______________________________________________ lava-users mailing list -- lava-users@lists.lavasoftware.org To unsubscribe send an email to lava-users-leave@lists.lavasoftware.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Hey there,
yeah, there are different ways to achieve this.
# PAT This seems to be the most difficult one: Git has to send an additional HTTP Authorization Header to fetch stuff: https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-pe...
So you would need to modify LAVA to do this.
Maybe it will work out if you use https://user:password@repo.example.com/ syntax for repo URL, with user being empty and password being your PAT (that's why there's a colon in the example above) Be careful, the PAT might show up in different places, I would not recommend using this way! I didn't test this.
# SSH Azure DevOps supports git via SSH - that's the easier way to go (even for daily development), put your public key on the server, and clone repo using this key.
This would work if you properly set up your .ssh/config on the LAVA server so you can clone the ADO repo. Drawback: With current LAVA you would have to do this for the root user of your server, as LAVA uses this to clone. I have a not yet finished MR open to use the lavaserver user instead: https://gitlab.com/lava/lava/-/merge_requests/2259 It is "not yet finished" because it doesn't pass the tests (as they are run on docker), but it is usable for a Debian install of LAVA. I will have to find time to polish this and get it merged.
Be careful: With PAT and SSH, on ADO it is not possible to restrict access to single repositories. So these credentials will have at least read access to all the repositories your ADO user has access to - with SSH even write access. Take caution!
# Gitolite (or other git server) So the safe way to go (security wise and fail-safety wise) is to set up a separate git server locally on your LAVA server. I use Debian package gitolite3 for this, having set up a separate LAVA user (cloning repos with SSH). I'm just pushing repositories that LAVA needs access to (and restrict LAVA user accordingly).
Development workflow is to push your local repo to this repository - and push to ADO repository for backup reasons.
Choose what fits your workflow and security considerations.
Best regards
Stefan
On 1/25/2024 9:48 AM, Milosz Wasilewski wrote:
Hi Michael,
I think the simple answer is that LAVA only supports git repositories without passwords. I think it might be possible to store the credentials in git global config on the dispatcher, but I haven't tried this. Under the hood the usual git client is used. If there is a way to configure credential manager for git not to ask any interactive questions, this might work.
Best Regards, Milosz
On Wed, Jan 24, 2024 at 10:34 PM michael.peddie@gallagher.com wrote:
Hi,
As stated in the subject, I was wondering if it is at all possible to use a repo on Azure DevOps instead of a GitHub one. I have been using a GitHub repo thus far, and it works just fine, but I need to move my files over to DevOps.
According to the docs here: https://docs.lavasoftware.org/lava/actions-test.html only git is supported at the moment, with URL and tar planned for. Would URL work for this if support came in at any point? Or would azure need its own support?
Or should git work and there is something wrong with the repo URL I am using? Because the particular error I get is that the password could not be read (I'm using a PAT in the link, like I did for GitHub)
Best Regards. Michael
Hi again,
I found out through research that it may be a project configuration issue that's stopping me from using a PAT, and as I can not change that I will try ssh until I either get it or give up.
When doing the ssh, are you sure it should be on the server and not the dispatcher? If it's no trouble could you explain the architecture to me? It was my understanding the dispatcher (in the case of using a qemu device) would deploy the device, boot with the image and log in, then (still the dispatcher) calls the git commands to clone and run the tests. Is this wrong?
Regardless I will be trying several configurations of ssh. The ones in mind are: - ssh key on server (one on root and one on lavaserver user), no config - ssh key on server (one on root and one on lavaserver user), with config These two I'm not sure about on root vs lava user, but I will likely try both as well there: - ssh key on dispatcher, no config - ssh key on dispatcher, with config
Best regards, Michael
Well, I did all that and made no progress. Keep getting 'Host key verification failed. fatal: Could not read from remote repository.' Whenever I run the git clone command just in the CLI on my lavaserver host (and on the dispatcher), I am required to put in the key password, might this have something to do with that? I don't know much about ssh keys and verification, apologies.
Until then, I have no idea where to go from here. For now I will keep working out of my GitHub repo, but any help to get this sorted would be greatly appreciated, so thank you in advance!
Best regards, Michael
Hey Michael,
okay, this one is an easy one: The "S" in SSH stands for secure - thus you have to bother about things like the host key. There's an background article for understanding the details: https://www.ssh.com/blog/what-are-ssh-host-keys It is so into my bones that I forgot to mention it, sorry.
When you run the git clone (or simply SSH with correct user) to your ADO server, you should (once) get a message that the authenticity can't be established and you should be able to acknowledge this. Then the host key will get added to .ssh/known_hosts (don't get confused if you can't find it in plain text there, this file is encrypted). On the next tries, you shouldn't get the message anymore. If you do a 'ssh -v user@ado.example.com' you should also see the line: debug1: Host 'ado.example.com' is known and matches the RSA host key.
For the key password: Yeah, that's another hurdle: You will have to save the key without the password (or create a new one without password if that's easier for you), because LAVA cannot put the password in. If git clone on shell works without further user interaction, it also should work with LAVA.
For your server/dispatcher question: I'm really not sure... My setup is so small that both run on the same machine and thus I don't see any difference.
Best regards Stefan
On 1/26/2024 2:36 AM, Michael Peddie wrote:
Well, I did all that and made no progress. Keep getting 'Host key verification failed. fatal: Could not read from remote repository.' Whenever I run the git clone command just in the CLI on my lavaserver host (and on the dispatcher), I am required to put in the key password, might this have something to do with that? I don't know much about ssh keys and verification, apologies.
Until then, I have no idea where to go from here. For now I will keep working out of my GitHub repo, but any help to get this sorted would be greatly appreciated, so thank you in advance!
Best regards, Michael _______________________________________________ lava-users mailing list -- lava-users@lists.lavasoftware.org To unsubscribe send an email to lava-users-leave@lists.lavasoftware.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
On Fri, Jan 26, 2024 at 7:09 AM Stefan lists.lavasoftware.org_23@green-sparklet.de wrote:
Hey Michael,
okay, this one is an easy one: The "S" in SSH stands for secure - thus you have to bother about things like the host key. There's an background article for understanding the details: https://www.ssh.com/blog/what-are-ssh-host-keys It is so into my bones that I forgot to mention it, sorry.
When you run the git clone (or simply SSH with correct user) to your ADO server, you should (once) get a message that the authenticity can't be established and you should be able to acknowledge this. Then the host key will get added to .ssh/known_hosts (don't get confused if you can't find it in plain text there, this file is encrypted). On the next tries, you shouldn't get the message anymore. If you do a 'ssh -v user@ado.example.com' you should also see the line: debug1: Host 'ado.example.com' is known and matches the RSA host key.
For the key password: Yeah, that's another hurdle: You will have to save the key without the password (or create a new one without password if that's easier for you), because LAVA cannot put the password in. If git clone on shell works without further user interaction, it also should work with LAVA.
For your server/dispatcher question: I'm really not sure... My setup is so small that both run on the same machine and thus I don't see any difference.
git clone runs on the dispatcher.
Best Regards, Milosz
Best regards Stefan
On 1/26/2024 2:36 AM, Michael Peddie wrote:
Well, I did all that and made no progress. Keep getting 'Host key verification failed. fatal: Could not read from remote repository.' Whenever I run the git clone command just in the CLI on my lavaserver host (and on the dispatcher), I am required to put in the key password, might this have something to do with that? I don't know much about ssh keys and verification, apologies.
Until then, I have no idea where to go from here. For now I will keep working out of my GitHub repo, but any help to get this sorted would be greatly appreciated, so thank you in advance!
Best regards, Michael _______________________________________________ lava-users mailing list -- lava-users@lists.lavasoftware.org To unsubscribe send an email to lava-users-leave@lists.lavasoftware.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
lava-users mailing list -- lava-users@lists.lavasoftware.org To unsubscribe send an email to lava-users-leave@lists.lavasoftware.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Hi Milosz and Stefan, thank you both for helping insofar,
I tried using a PAT again, and the script in the official Azure DevOps tutorial (can be found here: https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-pe...) works. It didn't previously, but it does now.
From my understanding it may be possible to write an inline test definition to launch a script (saved somewhere on the dispatcher I guess?), that clones the repo and launches the tests, is that right? Or is there a limitation to what an inline definition can do in regards to that?
Best Regards, Michael
lava-users@lists.lavasoftware.org
-
Michael Peddie -
michael.peddie＠gallagher.com
-
Milosz Wasilewski -
Stefan