8. Containerized deployment

Imagens simples do estivador
Pilhas de docas
- Enxame/doca-composição
  - Descrição da pilha
  - Limpeza
- Kubernetes
Implantação de nuvens
- AWS usecase

There are many ways to use containerized application, from the most simple (simple Docker images) to sophisticated (Kubernetes and so on).

Nota

Este tipo de implantação precisa da aplicação docker para ser instalado e executado. Verifique este `tutorial <https://www.docker.com/101-tutorial>>`_.

Dica

Docker run pre packaged application (aka images) which can be retrieved as sources (Dockerfile and resources) to build or already built from registries (private or public).

Nota

Os downloads de pacotes QGIS Debian-Ubuntu precisam de uma chave de autenticação gpg válida. Favor consultar as páginas de instalação para atualizar o seguinte arquivo Docker.

8.1. Imagens simples do estivador 

As the docker image does not exist in a public registry. you will need to build it. To do so create a directory qgis-server and within its directory:

create a file Dockerfile with this content:

FROM debian:bullseye-slim

ENV LANG=en_EN.UTF-8


RUN apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
        gnupg \
        ca-certificates \
        wget \
        locales \
    && localedef -i en_US -f UTF-8 en_US.UTF-8 \
    # Add the current key for package downloading
    # Please refer to QGIS install documentation (https://www.qgis.org/fr/site/forusers/alldownloads.html#debian-ubuntu)
    && mkdir -m755 -p /etc/apt/keyrings \
    && wget -O /etc/apt/keyrings/qgis-archive-keyring.gpg https://download.qgis.org/downloads/qgis-archive-keyring.gpg \
    # Add repository for latest version of qgis-server
    # Please refer to QGIS repositories documentation if you want other version (https://qgis.org/en/site/forusers/alldownloads.html#repositories)
    && echo "deb [signed-by=/etc/apt/keyrings/qgis-archive-keyring.gpg] https://qgis.org/debian bullseye main" | tee /etc/apt/sources.list.d/qgis.list \
    && apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
        qgis-server \
        spawn-fcgi \
        xauth \
        xvfb \
    && apt-get remove --purge -y \
        gnupg \
        wget \
    && rm -rf /var/lib/apt/lists/*

RUN useradd -m qgis

ENV TINI_VERSION v0.19.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini

ENV QGIS_PREFIX_PATH /usr
ENV QGIS_SERVER_LOG_STDERR 1
ENV QGIS_SERVER_LOG_LEVEL 2

COPY cmd.sh /home/qgis/cmd.sh
RUN chmod -R 777 /home/qgis/cmd.sh
RUN chown qgis:qgis /home/qgis/cmd.sh

USER qgis
WORKDIR /home/qgis

ENTRYPOINT ["/tini", "--"]

CMD ["/home/qgis/cmd.sh"]

criar um arquivo cmd.sh com este conteúdo:

#!/bin/bash

[[ $DEBUG == "1" ]] && env

exec /usr/bin/xvfb-run --auto-servernum --server-num=1 /usr/bin/spawn-fcgi -p 5555 -n -d /home/qgis -- /usr/lib/cgi-bin/qgis_mapserv.fcgi

construir a imagem com:

docker build -f Dockerfile -t qgis-server ./

8.1.1. Primeira execução 

Para executar o servidor, você precisará de um arquivo de projeto QGIS. Você pode utilizar um dos seus ou escolher esta amostra.

To do so, create a directory data within the directory qgis-server and copy your file in it. To comply with the following explanations, rename it to osm.qgs.

Nota

You may need to add advertised URLs under the QGIS Server tab of the Project ► Properties if the GetCapabilites are broken. For example if your server is exposed on port 8080, you will put this for advertised URL http://localhost:8080/qgis-server/. More information available in section Configurar seu projeto and subsequent.

Agora, você pode executar o servidor com:

docker network create qgis
docker run -d --rm --name qgis-server --net=qgis --hostname=qgis-server \
              -v $(pwd)/data:/data:ro -p 5555:5555 \
              -e "QGIS_PROJECT_FILE=/data/osm.qgs" \
              qgis-server

Opções utilizadas:

-d*: rodar em segundo plano
–rm: remover o container quando parar
–nome: nome do container a ser criado
–net*: (anteriormente criada) sub-rede
–hostname: nome da hostname do recipiente, para referência posterior
-v: diretório de dados local a ser montado no contêiner
**-p: mapeamento da porta
-e*: variável de ambiente a ser utilizada no contêiner

Para verificar, digite docker ps | grep qgis-server e você deve ver uma linha com qgis-server:

CONTAINER ID   IMAGE         COMMAND                  CREATED         STATUS         PORTS                    NAMES
4de8192da76e   qgis-server   "/tini -- /home/qgis…"   3 seconds ago   Up 2 seconds   0.0.0.0:5555->5555/tcp   qgis-server

8.1.2. Amostra utilizável 

As the server is only accepting fastcgi connections, you need an HTTP server that handles this protocol. To do so we have to create a simple Nginx configuration file and start a Nginx image.

Criar um arquivo nginx.conf no diretório atual com este conteúdo:

server {
  listen 80;
  server_name _;
  location / {
    root  /usr/share/nginx/html;
    index index.html index.htm;
  }
  location /qgis-server {
    proxy_buffers 16 16k;
    proxy_buffer_size 16k;
    gzip off;
    include fastcgi_params;
    fastcgi_pass qgis-server:5555;
  }
}

E digite este comando:

docker run -d --rm --name nginx --net=qgis --hostname=nginx \
              -v $(pwd)/nginx.conf:/etc/nginx/conf.d/default.conf:ro -p 8080:80 \
              nginx:1.13

To check capabilities availability, type in a browser http://localhost:8080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

8.1.3. Limpeza 

Para limpar as imagens em execução, digite:

docker stop qgis-server nginx

8.2. Pilhas de docas 

O método anterior é fácil de ser roteirizado, mas não pode ser facilmente compactado, nem padronizado ou facilmente administrado.

To work with a docker image set you could use a docker stack managed by an orchestrator. In a stack, the images are working in the same private network, and you can start / stop the whole stack or deploy the stack to other workers. There are many orchestrators, for example Swarm, Kubernetes and Mesos.

A seguir, apresentaremos configurações simples para fins de teste. Elas não são adequadas para produção.

8.2.1. Enxame/doca-composição 

Docker now has its own orchestrator: Swarm (compatible with docker-compose files). You have to enable it (the Mac version will also work with Linux).

Descrição da pilha 

Agora que o Swarm está funcionando, crie o arquivo de serviço (veja Deploy to Swarm) qgis-stack.yaml:

version: '3.7'

services:
  qgis-server:
    # Should use version with utf-8 locale support:
    image: qgis-server:latest
    volumes:
    - REPLACE_WITH_FULL_PATH/data:/data:ro
    environment:
    - LANG=en_EN.UTF-8
    - QGIS_PROJECT_FILE=/data/osm.qgs
    - QGIS_SERVER_LOG_LEVEL=0  # INFO (log all requests)
    - DEBUG=1                  # display env before spawning QGIS Server

  nginx:
    image: nginx:1.13
    ports:
    - 8080:80
    volumes:
    - REPLACE_WITH_FULL_PATH/nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
    - qgis-server

Para implantar (ou atualizar) a pilha, digite:

docker stack deploy -c qgis-stack.yaml qgis-stack

Check the stack deployment status until you obtain 1/1 in the replicas column:

docker stack services qgis-stack

Algo parecido com:

ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
gmx7ewlvwsqt        qgis_nginx          replicated          1/1                 nginx:1.13          *:8080->80/tcp
l0v2e7cl43u3        qgis_qgis-server    replicated          1/1                 qgis-server:latest

To check WMS capabilities, type in a web browser http://localhost:8080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

Limpeza 

Para limpeza, digite:

docker stack rm qgis-stack

8.2.2. Kubernetes 

Instalação 

If you have a Docker Desktop installation, using Kubernetes (aka k8s) is pretty straight forward: enable k8s.

If not, follow the minikube tutorial or microk8s for Ubuntu.

As Kubernetes installation can be really complex, we will only focus on aspects used by this demo. For further / deeper information, check the official documentation.

microk8s

microk8s needs extra steps: you have to enable the registry and tag the qgis-server image in order to have Kubernetes to find the created images.

Primeiro, habilite o registro:

microk8s enable dashboard dns registry

Em seguida, etiquete e passe a imagem para seu registro recém-criado:

docker tag qgis-server 127.0.0.1:32000/qgis-server && docker push 127.0.0.1:32000/qgis-server

Finally, add or complete the /etc/docker/daemon.json to have your registry 127.0.0.1:32000 listed in the insecure-registries field:

{
  "insecure-registries": ["127.0.0.1:32000"]
}

Criando manifestos 

Kubernetes describes the objects to deploy in yaml manifests. There are many different kinds, but we will only use deployments (handle pods, i.e. docker images) and services to expose the deployments to internal or external purposes.

Manifestações de implantação

Criar um arquivo :arquivo:`deployments.yaml` com este conteúdo:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: qgis-server
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      myLabel: qgis-server
  template:
    metadata:
      labels:
        myLabel: qgis-server
    spec:
      containers:
        - name: qgis-server
          image: localhost:32000/qgis-server:latest
          imagePullPolicy: IfNotPresent
          env:
            - name: LANG
              value: en_EN.UTF-8
            - name: QGIS_PROJECT_FILE
              value: /data/osm.qgs
            - name: QGIS_SERVER_LOG_LEVEL
              value: "0"
            - name: DEBUG
              value: "1"
          ports:
            - containerPort: 5555
          volumeMounts:
            - name: qgis-data
              mountPath: /data/
      volumes:
        - name: qgis-data
          hostPath:
            path: REPLACE_WITH_FULL_PATH/data

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: qgis-nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      myLabel: qgis-nginx
  template:
    metadata:
      labels:
        myLabel: qgis-nginx
    spec:
      containers:
        - name: qgis-nginx
          image: nginx:1.13
          ports:
            - containerPort: 80
          volumeMounts:
            - name: nginx-conf
              mountPath: /etc/nginx/conf.d/default.conf
      volumes:
        - name: nginx-conf
          hostPath:
            path: REPLACE_WITH_FULL_PATH/nginx.conf

Manifestos de serviço

Criar um arquivo :arquivo:`services.yaml` com este conteúdo:

apiVersion: v1
kind: Service
metadata:
  name: qgis-server
  namespace: default
spec:
  type: ClusterIP
  selector:
    myLabel: qgis-server
  ports:
    - port: 5555
      targetPort: 5555
---
apiVersion: v1
kind: Service
metadata:
  name: qgis-nginx
  namespace: default
spec:
  type: NodePort
  selector:
    myLabel: qgis-nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080

Implantação de manifestos 

Para implantar as imagens e serviços em Kubernetes, pode-se usar o painel de controle (clique no +* no canto superior direito) ou a linha de comando.

Nota

When using the command line with microk8s you will have to prefix each command with microk8s.

Para implantar ou atualizar seus manifestos:

kubectl apply -k ./

Para verificar o que está atualmente implantado:

kubectl get pods,services,deployment

Você deveria obter algo como:

NAME                               READY   STATUS    RESTARTS   AGE
pod/qgis-nginx-54845ff6f6-8skp9    1/1     Running   0          27m
pod/qgis-server-75df8ddd89-c7t7s   1/1     Running   0          27m

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service/Kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP        5h51m
service/qgis-exec-server   ClusterIP   10.152.183.218   <none>        5555/TCP       35m
service/qgis-nginx         NodePort    10.152.183.234   <none>        80:30080/TCP   27m
service/qgis-server        ClusterIP   10.152.183.132   <none>        5555/TCP       27m

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/qgis-nginx    1/1     1            1           27m
deployment.apps/qgis-server   1/1     1            1           27m

Para ler os logs nginx/qgis, digite:

kubectl logs -f POD_NAME

To check WMS capabilities, type in a web browser http://localhost:30080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

Limpeza 

Para limpar, digite:

kubectl delete -n default service/qgis-server service/qgis-nginx deployment/qgis-nginx deployment/qgis-server

8.3. Implantação de nuvens 

Managing your own cluster of servers to handle the deployment of containerized applications, is a complex job. You have to handle multiple issues, such as hardware, bandwidth and security at different levels.

Cloud deployment solutions can be a good alternative when you do not want to focus on infrastructure management.

A cloud deployment may use proprietary mechanisms, but they are also compatible with the stages explained previously (docker images and stack management).

8.3.1. AWS usecase 

With Amazon AWS, through ECS (Elastic Container Service) functionalities, you can use docker-compose or Kubernetes compatible wrappers to manage your stack. You will have to create an image registry for your custom images to be accessible.

To use docker-compose alike functionalities, you need to install the ecs-cli client and have proper permissions / roles. Then, with the help of the ecs-cli compose commands, you can reuse the stack description.

To use Kubernetes, you can use the AWS web console or the command line tool eksctl and have the proper permissions / roles. Then with a well configured kubectl environment, you can reuse the Kubernetes manifests.