On Wed, 16 Jan 2019 at 16:56, Olli Vainola Olli.Vainola@arm.com wrote:

Hi,

I’ve been using the lava-server docker image (hub.lavasoftware.org/lava/lava/lava-server:2018.10) for a couple of weeks and I had some problems on making the data persistent for the postgres (mapping the volume from host to container). Anyways I decided to take postgres out from the startup by modifying the entrypoint.sh file:

++++++++++++++ Added these lines after start_lava_server_gunicorn -function ++++++++++++++++++++++++++

if [[ ! -z $DJANGO_POSTGRES_SERVER ]]; then

txt="s/LAVA_DB_SERVER=\"localhost\"/LAVA_DB_SERVER=\"$DJANGO_POSTGRES_SERVER\"/g"

sed -i $txt /etc/lava-server/instance.conf

fi

if [[ ! -z $DJANGO_POSTGRES_PORT ]]; then

txt="s/LAVA_DB_PORT=\"5432\"/LAVA_DB_PORT=\"$DJANGO_POSTGRES_PORT\"/g"

sed -i $txt /etc/lava-server/instance.conf

fi

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

---------------------------- Commented out these lines -----------------------------

# Start all services

#echo "Starting postgresql"

#/etc/init.d/postgresql start

#echo "done"

#echo

#echo "Waiting for postgresql"

#wait_postgresql

#echo "[done]"

#echo

---

After that I created a new Dockerfile and built a new image:

+++++++++++++++++++++++++ Dockerfile +++++++++++++++++++++++

FROM hub.lavasoftware.org/lava/lava/lava-server:2018.10

COPY ./entrypoint.sh /root/

RUN chmod 755 /root/entrypoint.sh

ENTRYPOINT ["/root/entrypoint.sh"]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

...

...

sudo docker build -t lava-server:mod .

Just to get all up and running I made a docker-compose.yml file to kick postgres and lava-server up

+++++++++++++++++++++++++++ docker-compose.yml +++++++++++++++++++++++++

version: '3'

services:

postgres:

image: postgres:11

restart: always

environment:

  POSTGRES_DB: lavaserver

  POSTGRES_USER: lavaserver

  POSTGRES_PASSWORD: d3e5d13fa15f

lava-server:

depends_on:

 - postgres

image: lava-server:mod

environment:

  DJANGO_POSTGRES_PORT: 5432

  DJANGO_POSTGRES_SERVER: postgres

ports:

 - "80:80"

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

I still feel that there is too much going on inside that lava-server:mod image, since it has following softwares running:

Lavamaster Gunicorn Logger Publisher

Olli, Dan already did part of what you propose: https://github.com/danrue/lava-docker-compose I guess we should move the work to some common place not to repeat all the same steps

milosz

What do you think, should I still break it into smaller pieces? Pros on this would be that softwares wouldn’t die silently (started by using ‘&’ and docker could try to restart them), logs would be in their own logs windows’ (docker log CONTAINER) and containers themselves would be more configurable via environment variables.

Br

Olli Väinölä

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users

Hi,

-----Original Message----- From: Dan Rue dan.rue@linaro.org Sent: Wednesday, January 16, 2019 10:46 PM To: Milosz Wasilewski milosz.wasilewski@linaro.org Cc: Olli Vainola Olli.Vainola@arm.com; lava-users@lists.lavasoftware.org Subject: Re: [Lava-users] Splitting the lava-server docker image

On Wed, Jan 16, 2019 at 07:27:28PM +0000, Milosz Wasilewski wrote:

...

On Wed, 16 Jan 2019 at 16:56, Olli Vainola Olli.Vainola@arm.com wrote:

...

Hi,

I’ve been using the lava-server docker image (hub.lavasoftware.org/lava/lava/lava-server:2018.10) for a couple of weeks and I had some problems on making the data persistent for the postgres (mapping the volume from host to container). Anyways I decided to take postgres out from the startup by modifying the entrypoint.sh file:

++++++++++++++ Added these lines after start_lava_server_gunicorn ++++++++++++++ -function ++++++++++++++++++++++++++

if [[ ! -z $DJANGO_POSTGRES_SERVER ]]; then

txt="s/LAVA_DB_SERVER=\"localhost\"/LAVA_DB_SERVER=\"$DJANGO_POSTGRES_SERVER\"/g"

sed -i $txt /etc/lava-server/instance.conf

fi

if [[ ! -z $DJANGO_POSTGRES_PORT ]]; then

txt="s/LAVA_DB_PORT=\"5432\"/LAVA_DB_PORT=\"$DJANGO_POSTGRES_PORT\"/g"

sed -i $txt /etc/lava-server/instance.conf

fi

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

---------------------------- Commented out these lines

# Start all services

#echo "Starting postgresql"

#/etc/init.d/postgresql start

#echo "done"

#echo

#echo "Waiting for postgresql"

#wait_postgresql

#echo "[done]"

#echo

---

---

After that I created a new Dockerfile and built a new image:

+++++++++++++++++++++++++ Dockerfile +++++++++++++++++++++++

FROM hub.lavasoftware.org/lava/lava/lava-server:2018.10

COPY ./entrypoint.sh /root/

RUN chmod 755 /root/entrypoint.sh

ENTRYPOINT ["/root/entrypoint.sh"]

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

...

...

sudo docker build -t lava-server:mod .

Just to get all up and running I made a docker-compose.yml file to kick postgres and lava-server up

+++++++++++++++++++++++++++ docker-compose.yml +++++++++++++++++++++++++++ +++++++++++++++++++++++++

version: '3'

services:

postgres:

image: postgres:11

restart: always

environment:

  POSTGRES_DB: lavaserver

  POSTGRES_USER: lavaserver

  POSTGRES_PASSWORD: d3e5d13fa15f

lava-server:

depends_on:

 - postgres

image: lava-server:mod

environment:

  DJANGO_POSTGRES_PORT: 5432

  DJANGO_POSTGRES_SERVER: postgres

ports:

 - "80:80"

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

I still feel that there is too much going on inside that lava-server:mod image, since it has following softwares running:

Lavamaster Gunicorn Logger Publisher

Olli, Dan already did part of what you propose: https://github.com/danrue/lava-docker-compose I guess we should move the work to some common place not to repeat all the same steps

Yes, it sounds like Olli and I are on the same path :)

I didn't bother modifying the container though. I just mounted in my own copy of instance.conf. Adding environment variable support, as you have proposed, for common config properties would be better. This meant that postgresql is still running an empty database in my server container for the time being.

I was also thinking on injecting the configuration file via volume mount, but then again postgres was running there for no reason, so I figured just to modify the entrypoint.sh itself :). A bit hacky hacky but did the trick.

By the way, I noticed you're using postgres 11, yet the lava-server container uses 9.6. It's probably fine? I have no idea - I tried to keep the version consistent with what's tested and supported. Also, in case you need it, there was a fix to the entrypoint to properly wait for postgres. Find the latest entrypoint at https://git.lavasoftware.org/lava/pkg/docker/blob/master/amd64/lava-server/e...

At least the data migration part went ok, so I'll go with a strong maybe. Web interface responded properly but I didn't add slaves yet, so haven't tested it fully. I also found that entrypoint.sh from the gitlab and yes, there was some timing issues when postgres didn't start fast enough and lava-server crashed. Rather than wait, there could be some sanity check for connection with some retry & timeout functionality.

...

...

What do you think, should I still break it into smaller pieces? Pros on this would be that softwares wouldn’t die silently (started by using ‘&’ and docker could try to restart them), logs would be in their own logs windows’ (docker log CONTAINER) and containers themselves would be more configurable via environment variables.

I agree in principle, but I'm not sure of the implications for each of the services. For the time being, I'm focusing on trying to use the containers as they are, and upstream improvements to make them easier to use.

I can try to separate all server side services and see if any major issues arise.

I do think I'll try using a separate ser2net container with my dispatcher though - it seems like an easy case, ser2net's already not started by default, and I want a newer version of ser2net than what's installed in the dispatcher by default.

Olli - in light of all that, I'm curious about your feedback on this merge request: https://git.lavasoftware.org/lava/pkg/docker/merge_requests/10.

Looks great! That would make the starting the container more generic and one could add more functionality without building the container.

Br Olli

Thanks! Dan

...

...

Br

Olli Väinölä

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users

---

Lava-users mailing list Lava-users@lists.lavasoftware.org https://lists.lavasoftware.org/mailman/listinfo/lava-users

-- Linaro - Kernel Validation

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.