lava-users

lava-users@lists.lavasoftware.org

1 participants
930 discussions

Re: [Lava-users] LAVA master backup/restore

by Philippe Mazet (OSS)

Ok. You get 2 MRs for the price of one : In docker-compose tree: https://git.lavasoftware.org/lava/pkg/docker-compose/-/merge_requests/37 In LAVA documentation: https://git.lavasoftware.org/lava/lava/-/merge_requests/1479 May I ask an update about our other pending MRs? ;-) Thanks a lot in advance, Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Wednesday, April 14, 2021 9:20 AM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Hello, if you have some time, a patch to update the documentation would be greatly appreciated !! Thanks a lot. Le ven. 2 avr. 2021 à 11:34, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Hi Rémi, I finally found this to work: For creating the backup, run backup command as postgres user, eg: docker exec --user postgres docker-compose_db_1 bash -c "pg_dump --username=lavaserver lavaserver > /var/lib/postgresql/data/lavaserver.sql" Then for restoring backup, place the sql file in a folder that will be mounted as a volume in postgres container, on /docker-entrypoint-initdb.d/. On way to do this is to create a dedicated docker-compose file (docker-compose-restore-backup.yaml) that will be appended to docker-compose.yaml: version: "3.4" services: db: volumes: - ./initdb.d:/docker-entrypoint-initdb.d Then: docker-compose stop docker container rm docker-compose_db_1; docker volume rm lava-server-pgdata docker-compose -f docker-compose.yaml -f docker-compose-restore-backup.yaml up -d I hope that helps. Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Friday, March 26, 2021 4:56 PM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Hello Philippe, I haven't tried so I don't have the detailed procedure. When you find out, could you share the procedure ? Thanks Le mar. 23 mars 2021 à 11:28, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Thanks Rémi, but result is the same when I start only the db container, even after a full cleanup (docker container rm docker-compose_db_1 && docker volume rm lava-server-pgdata): ERROR: database "lavaserver" already exists Seems to be related somehow to "POSTGRES_USER: lavaserver" in docker-compose.yaml. Reading the entrypoint script for postgres (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh), I thought using PGUSER could help, but I didn't manage yet to get that working. I will continue digging into this... but if you have a detailed procedure somewhere, please share 😊 Thanks a lot! Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Monday, March 22, 2021 9:25 AM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Le jeu. 18 mars 2021 à 16:22, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Hi all, I am using the docker-compose solution to run a master: https://git.lavasoftware.org/lava/pkg/docker-compose I was wondering how to extract a backup of the DB, and re-inject it in a different instance. I found that postgres image has its own "restore" mechanism, based on initdb.d folder, as mentioned in the documentation: https://registry.hub.docker.com/_/postgres/ The entrypoint.sh script (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh) handles the restore whenever the folder is /docker-entrypoint-initdb.d/ contains a .sql file. But when we store a backup in the container's /docker-entrypoint-initdb.d/ folder, and remove both the postgres image and its db-data volume, we get this error on next start: ERROR: database "lavaserver" already exists Full startup log attached. Questions: - Does Linaro's docker-compose solution modify postgres restore mechanism in any way? I guess that's only because lava will automatically create an empty lavaserver database. So in your use case, you should only start the db service and not any lava-* services. When the restoration has been done, you can restart every lava services. - How do you handle backup/restore at linaro? Thanks a lot in advance, Philippe Mazet NXP Semiconductors - Edge Processing Email: philippe.mazet(a)nxp.com _______________________________________________ Lava-users mailing list Lava-users(a)lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users -- Rémi Duraffort LAVA Architect Linaro -- Rémi Duraffort LAVA Architect Linaro -- Rémi Duraffort TuxArchitect Linaro

4 years, 4 months

LAVA exporting bash shell while using debian OS parameter

by P T, Sarath

Hi Team, I have used the LAVA job definition including deployment data(OS parameter) as debian . As per the deployment_data.py it should export the shell as /bin/bash but it's exporting /bin/sh. I am observing this thing after updating LAVA to 2021.01. Below I am giving the job definition file which I used. It will be very helpful if anyone reply on this. Thank you . Job definition ============= device_type: x86-simatic-slll job_name: x86-simatic-ipc227e-slll health-check timeouts: job: minutes: 20 action: minutes: 20 connection: minutes: 10 priority: medium visibility: public tags: - slll-simatic-ipc-227e-01 actions: - deploy: to: overlay - boot: method: minimal reset: true failure_retry: 2 auto_login: login_prompt: 'ebsy-isar login:' username: root password_prompt: 'Password:' password: root prompts: - root@ebsy-isar:~# transfer_overlay: download_command: wget unpack_command: tar -C / -xzf # TEST_BLOCK - test: timeout: minutes: 5 definitions: - repository: metadata: format: Lava-Test Test Definition 1.0 name: kernel-version-basic description: "check kernel version" os: - debian scope: - functional run: steps: - lava-test-case uname --shell uname -a from: inline name: kernel-version-inline path: inline/kernel-version-basic.yaml lava-signal: kmsg Regards Sarath P T

4 years, 4 months

Re: [Lava-users] LAVA master backup/restore

by Philippe Mazet (OSS)

Hi Rémi, I finally found this to work: For creating the backup, run backup command as postgres user, eg: docker exec --user postgres docker-compose_db_1 bash -c "pg_dump --username=lavaserver lavaserver > /var/lib/postgresql/data/lavaserver.sql" Then for restoring backup, place the sql file in a folder that will be mounted as a volume in postgres container, on /docker-entrypoint-initdb.d/. On way to do this is to create a dedicated docker-compose file (docker-compose-restore-backup.yaml) that will be appended to docker-compose.yaml: version: "3.4" services: db: volumes: - ./initdb.d:/docker-entrypoint-initdb.d Then: docker-compose stop docker container rm docker-compose_db_1; docker volume rm lava-server-pgdata docker-compose -f docker-compose.yaml -f docker-compose-restore-backup.yaml up -d I hope that helps. Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Friday, March 26, 2021 4:56 PM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Hello Philippe, I haven't tried so I don't have the detailed procedure. When you find out, could you share the procedure ? Thanks Le mar. 23 mars 2021 à 11:28, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Thanks Rémi, but result is the same when I start only the db container, even after a full cleanup (docker container rm docker-compose_db_1 && docker volume rm lava-server-pgdata): ERROR: database "lavaserver" already exists Seems to be related somehow to "POSTGRES_USER: lavaserver" in docker-compose.yaml. Reading the entrypoint script for postgres (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh), I thought using PGUSER could help, but I didn't manage yet to get that working. I will continue digging into this... but if you have a detailed procedure somewhere, please share 😊 Thanks a lot! Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Monday, March 22, 2021 9:25 AM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Le jeu. 18 mars 2021 à 16:22, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Hi all, I am using the docker-compose solution to run a master: https://git.lavasoftware.org/lava/pkg/docker-compose I was wondering how to extract a backup of the DB, and re-inject it in a different instance. I found that postgres image has its own "restore" mechanism, based on initdb.d folder, as mentioned in the documentation: https://registry.hub.docker.com/_/postgres/ The entrypoint.sh script (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh) handles the restore whenever the folder is /docker-entrypoint-initdb.d/ contains a .sql file. But when we store a backup in the container's /docker-entrypoint-initdb.d/ folder, and remove both the postgres image and its db-data volume, we get this error on next start: ERROR: database "lavaserver" already exists Full startup log attached. Questions: - Does Linaro's docker-compose solution modify postgres restore mechanism in any way? I guess that's only because lava will automatically create an empty lavaserver database. So in your use case, you should only start the db service and not any lava-* services. When the restoration has been done, you can restart every lava services. - How do you handle backup/restore at linaro? Thanks a lot in advance, Philippe Mazet NXP Semiconductors - Edge Processing Email: philippe.mazet(a)nxp.com _______________________________________________ Lava-users mailing list Lava-users(a)lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users -- Rémi Duraffort LAVA Architect Linaro -- Rémi Duraffort LAVA Architect Linaro

4 years, 4 months

Re: [Lava-users] Worker/master version mismatch

by Philippe Mazet (OSS)

I have submitted a MR to allow version mismatch: https://git.lavasoftware.org/lava/lava/-/merge_requests/1466 This feature is really a *must-have* for us. At our own risks of course. Thanks for considering it! We also have quite a few other MRs in the pipe. Some of them are few months old, even if all comments are replied. What is the appropriate way to get them reviewed/merged? ;-) Thanks a lot, Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Tuesday, January 26, 2021 10:04 AM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] Worker/master version mismatch Hello Philippe, I tried the lava-docker-worker but found it too restrictive, since we can’t customize the Docker container at all (adding some tools, cfg files, ssh keys, etc…), and was imposing usage of Docker within the job themselves (that would force us to rewrite many of our existing tests). That's an interesting use case. Maybe something that we might want to support. A possibility would be to specify the docker registry that you want to use instead of docker hub. As to “automatically run the right worker version”, I might have missed something. By checking lava-docker-worker sources, I concluded the Docker would simply refuse to start if version differs versus master. But do you mean it will automatically restart with the correct version when master is upgraded? That would be an easy fix (just adding a settings to lava_server and return depending on the setting value). But keep in mind that this might break at any release. Rgds -- Rémi Duraffort LAVA Architect Linaro

4 years, 4 months

We use “run_step” method in lava definitions, but it cannot work when we upgrade lava from 2018.11 stretch to 2021.01

by yangjianxiang

Hi ： We use “run_step” method in lava definitions, but it cannot work when we upgrade lava from 2018.11 stretch to 2021.01. How can we slove this problem. [cid:image001.png@01D73098.D3FE9C80]

4 years, 4 months

About device change permission.

by Larry Shen

Hi, guys, I'm currently debug permission issue on our production master, it looks like next: 1. I set user e.g. "zz" to group "imxcn", and assign device "imx8mm-evk-sh99" with the permission "change_device". But, in our production master, "zz"(not admin), can't change the device status. What's strange is: I have a local master on my hand, on this master, I do the same configure, the non-admin user can change the device status. 2. I check the database of our production master as next; auth_user: 39 | | 2021-03-30 08:37:32.98708+00 | f | zz auth_group: 8 | imxcn auth_user_groups: 290 | 39 | 8 lava_scheduler_app_groupdevicepermission: 91 | imx8mm-evk-sh99 | 8 | 9 92 | imx8mm-evk-sh99 | 8 | 5 auth_permission: 5 | Can submit jobs to device | 2 | submit_to_device 9 | Can change device | 2 | change_device django_content_type: 2 | lava_scheduler_app | device Looks all data is ok, so, base on above data, do you know any possible reason why our production server can't allow non-admin user change device permission? I'm a little lost as my local on hand server works, just central production server not work... Thanks.

4 years, 4 months

How to change device healthy using rest api?

by cnspring2002

Hello, LAVA maintainer: I used to use xmlrpc to change the status of our device, now I want to switch to rest api, but I don't know how to handle it with next: https://validation.linaro.org/static/docs/v2/data-export.html#v0-2 It didn't give an example, and also did not specify using GET, POST, PUT or DELETE, could you give an example to me? Like a curl command format. Thank you!

4 years, 4 months

Suggestion on avoiding cloning of same repositories multiple times.

by Kumar, Maanas

Hello, We have job definition with multiple test definitions as shown below. test: timeout: hours: 4 definitions: - repository: ssh://git@github.com:7999/<MytestRepo>.git from: git path: automated/linux/lttng/lttng.yaml name: lttng - repository: ssh://git@github.com:7999/<MytestRepo> from: git path: automated/linux/docker/docker.yaml name: docker We want to avoid multiple cloning of same repository as all our test-definitions are in one repo. We require suggestions on how to avoid cloning of same repository multiple times.? Regards, Maanas M D

4 years, 4 months

Re: [Lava-users] LAVA master backup/restore

by Philippe Mazet (OSS)

Thanks Rémi, but result is the same when I start only the db container, even after a full cleanup (docker container rm docker-compose_db_1 && docker volume rm lava-server-pgdata): ERROR: database "lavaserver" already exists Seems to be related somehow to "POSTGRES_USER: lavaserver" in docker-compose.yaml. Reading the entrypoint script for postgres (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh), I thought using PGUSER could help, but I didn't manage yet to get that working. I will continue digging into this... but if you have a detailed procedure somewhere, please share 😊 Thanks a lot! Philippe From: Lava-users <lava-users-bounces(a)lists.lavasoftware.org> On Behalf Of Remi Duraffort Sent: Monday, March 22, 2021 9:25 AM To: Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> Cc: lava-users(a)lists.lavasoftware.org <lava-users(a)lavasoftware.org> Subject: Re: [Lava-users] LAVA master backup/restore Le jeu. 18 mars 2021 à 16:22, Philippe Mazet (OSS) <philippe.mazet(a)oss.nxp.com> a écrit : Hi all, I am using the docker-compose solution to run a master: https://git.lavasoftware.org/lava/pkg/docker-compose I was wondering how to extract a backup of the DB, and re-inject it in a different instance. I found that postgres image has its own "restore" mechanism, based on initdb.d folder, as mentioned in the documentation: https://registry.hub.docker.com/_/postgres/ The entrypoint.sh script (https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh) handles the restore whenever the folder is /docker-entrypoint-initdb.d/ contains a .sql file. But when we store a backup in the container's /docker-entrypoint-initdb.d/ folder, and remove both the postgres image and its db-data volume, we get this error on next start: ERROR: database "lavaserver" already exists Full startup log attached. Questions: - Does Linaro's docker-compose solution modify postgres restore mechanism in any way? I guess that's only because lava will automatically create an empty lavaserver database. So in your use case, you should only start the db service and not any lava-* services. When the restoration has been done, you can restart every lava services. - How do you handle backup/restore at linaro? Thanks a lot in advance, Philippe Mazet NXP Semiconductors - Edge Processing Email: philippe.mazet(a)nxp.com _______________________________________________ Lava-users mailing list Lava-users(a)lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users -- Rémi Duraffort LAVA Architect Linaro

4 years, 5 months

docker support for adb-fastboot

by Marcel Trattner

Dear LAVA-Users, as far as we know the lxc support for LAVA test is going to be depricated and docker should be used instead, so we tried to define a test job with docker. The lxc job for this device was completing but when we tried to follow this tutorial: https://www.youtube.com/watch?v=y-5DjzQztAk with the following test job: timeouts: job: minutes: 30 connection: minutes: 2 actions: finalize: seconds: 60 context: test_character_delay: 10 device_type: dragonboard_845_uSOM job_name: Docker-Test priority: 50 visibility: public metadata: build-url: '' actions: - deploy: to: fastboot docker: image: adb-fastboot timeout: minutes: 15 images: boot: url: https://path/to/image/boot.img reboot: hard-reset - boot: method: fastboot docker: image: adb-fastboot prompts: - root@(.*):/# timeout: minutes: 15 - test: docker: image: adb-fastboot timeout: minutes: 10 definitions: - repository: https://github.com/Linaro/test-definitions.git from: git path: automated/linux/android-platform-tools/install.yaml name: install-android-platform-tools-r2800 parameters: LINK: https://dl.google.com/android/repository/platform-tools_r28.0.0-linux.zip We got this error: [cid:image001.jpg@01D7224A.65849130] Are we doing something wrong or is this a bug? Regards, Marcel _____________________________________________________________________________________ Expleo Germany GmbH Sitz der Gesellschaft | Corporate Headquarters: M?nchen Handelsregister | Commercial Register: Amtsgericht M?nchen HRB 83252 Gesch?ftsf?hrung | Management: Ralph Gillessen, Marcus Ganguin This message contains information that may be privileged or confidential. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. _________________________________________________________ EMAIL LEGAL MENTION / DISCLAIMER This message contains information that may be privileged or confidential and is the property of the Expleo Services SAS, RCS Versailles 831 178 348, located, 3 avenue des Pr?s, 78180 Montigny Le Bretonneux - France. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message. Ce message contient des informations qui peuvent ?tre privil?gi?es ou confidentielles et elles sont de la propri?t? d'Expleo Services SAS, RCS Versailles 831 178 348, situ?, 3 avenue des Pr?s, 78180 Montigny le Bretonneux-France. Il est destin? uniquement ? la personne ? qui est adress?. Si vous n'?tes pas le destinataire vis?, vous n'?tes pas autoris? ? lire, imprimer, conserver, copier, diffuser, distribuer ou utiliser ce message ou toute partie de celui-ci. Si vous recevez ce message par erreur, veuillez en avertir imm?diatement l'exp?diteur et supprimer toutes les copies de ce message.

4 years, 5 months

← Newer
1
...
15
16
17
18
19
20
21
...
93
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

lava-users