7. Kontejnerové nasazení

Jednoduché docker images
Docker stacks
- Swarm/docker-compose
  - Popis Stacku
  - Cleanup
- Kubernetes
Cloud deployment
- AWS usecase

Existuje mnoho způsobů, jak používat kontejnerovou aplikaci, od nejjednodušších (jednoduché sestavení Dockeru) po sofistikované (Kubernetes a tak dále).

Poznámka

Tento druh nasazení vyžaduje, aby byla nainstalována a spuštěna aplikace docker <https://www.docker.com> _. Podívejte se na tento tutoriál <https://www.docker.com/101-tutorial> _.

Rada

Docker spouští předbalenou aplikaci (tzv. images), kterou lze načíst jako zdroje (Dockerfile a prostředky) k vytvoření nebo již vytvořené z registrů (soukromých nebo veřejných).

Poznámka

QGIS Debian-Ubuntu package downloads need a valid gpg authentication key. Please refer to the installation pages to update the following Dockerfile with the latest key fingerprint

7.1. Jednoduché docker images 

Protože docker image ve veřejném registru neexistuje. budete to muset postavit. Chcete-li tak učinit, vytvořte adresář: file: qgis-server a v jeho adresáři:

vytvořte soubor :file: Dockerfile s tímto obsahem:

FROM debian:buster-slim

ENV LANG=en_EN.UTF-8


RUN apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
        gnupg \
        ca-certificates \
        wget \
        locales \
    && localedef -i en_US -f UTF-8 en_US.UTF-8 \
    # Add the current key for package downloading - As the key changes every year at least
    # Please refer to QGIS install documentation and replace it with the latest one
    && wget -O - https://qgis.org/downloads/qgis-2020.gpg.key | gpg --import \
    && gpg --export --armor F7E06F06199EF2F2 | apt-key add - \
    && echo "deb http://qgis.org/debian buster main" >> /etc/apt/sources.list.d/qgis.list \
    && apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
        qgis-server \
        spawn-fcgi \
        xauth \
        xvfb \
    && apt-get remove --purge -y \
        gnupg \
        wget \
    && rm -rf /var/lib/apt/lists/*

RUN useradd -m qgis

ENV TINI_VERSION v0.17.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini

ENV QGIS_PREFIX_PATH /usr
ENV QGIS_SERVER_LOG_STDERR 1
ENV QGIS_SERVER_LOG_LEVEL 2

COPY cmd.sh /home/qgis/cmd.sh
RUN chmod -R 777 /home/qgis/cmd.sh
RUN chown qgis:qgis /home/qgis/cmd.sh

USER qgis
WORKDIR /home/qgis

ENTRYPOINT ["/tini", "--"]

CMD ["/home/qgis/cmd.sh"]

vytvořte soubor :file: cmd.sh s tímto obsahem:

#!/bin/bash

[[ $DEBUG == "1" ]] && env

exec /usr/bin/xvfb-run --auto-servernum --server-num=1 /usr/bin/spawn-fcgi -p 5555 -n -d /home/qgis -- /usr/lib/cgi-bin/qgis_mapserv.fcgi

vytvořit image pomocí:

docker build -f Dockerfile -t qgis-server ./

7.1.1. First run 

To run the server you will need a QGIS project file. You can use one of yours or pick this sample.

Chcete-li tak učinit, vytvořte adresář data v adresáři qgis-server a zkopírujte do něj soubor. Chcete-li dodržet následující vysvětlení, přejmenujte jej na osm.qgs.

Poznámka

You may need to add advertised URLs under the QGIS Server tab of the Project ► Properties if the GetCapabilites are broken. For example if your server is exposed on port 8080, you will put this for advertised URL http://localhost:8080/qgis-server/. More information available in section Configure your project and subsequent.

Nyní můžete server spustit pomocí:

docker network create qgis
docker run -d --rm --name qgis-server --net=qgis --hostname=qgis-server \
              -v $(pwd)/data:/data:ro -p 5555:5555 \
              -e "QGIS_PROJECT_FILE=/data/osm.qgs" \
              qgis-server

Použité možnosti:

-d: běží na pozadí
–rm: remove the container when it is stopped
–name: name of the container to be created
–net: (previously created) sub network
–hostname: container hostname, for later referencing
-v: local data directory to be mounted in the container
-p: host/container port mapping
-e: environment variable to be used in the container

To check, type docker ps | grep qgis-server and you should see a line with qgis-server:

CONTAINER ID   IMAGE         COMMAND                  CREATED         STATUS         PORTS                    NAMES
4de8192da76e   qgis-server   "/tini -- /home/qgis…"   3 seconds ago   Up 2 seconds   0.0.0.0:5555->5555/tcp   qgis-server

7.1.2. Použitelný vzorek 

As the server is only accepting fastcgi connections, you need an HTTP server that handles this protocol. To do so we have to create a simple Nginx configuration file and start a Nginx image.

Create a file nginx.conf in the current directory with this content:

server {
  listen 80;
  server_name _;
  location / {
    root  /usr/share/nginx/html;
    index index.html index.htm;
  }
  location /qgis-server {
    proxy_buffers 16 16k;
    proxy_buffer_size 16k;
    gzip off;
    include fastcgi_params;
    fastcgi_pass qgis-server:5555;
  }
}

A zadejte tento příkaz:

docker run -d --rm --name nginx --net=qgis --hostname=nginx \
              -v $(pwd)/nginx.conf:/etc/nginx/conf.d/default.conf:ro -p 8080:80 \
              nginx:1.13

Chcete-li zkontrolovat dostupnost funkcí, zadejte do prohlížeče http://localhost:8080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

7.1.3. Cleanup 

To cleanup the running images, type:

docker stop qgis-server nginx

7.2. Docker stacks 

Předchozí metoda je skriptovatelná, ale není snadno zabalitelná, standardizovaná nebo snadno spravovatelná.

Chcete-li pracovat se sadou image dockeru, můžete použít docker stack spravovaný orchestrátorem. Ve stacku image fungují ve stejné privátní síti a můžete spustit / zastavit celý stack nebo nasadit stack na jiné pracovníky. Existuje mnoho orchestrátorů, například Swarm, Kubernetes a Mesos.

V následujícím textu představíme jednoduché konfigurace pro účely testování. Nejsou vhodné pro výrobu.

7.2.1. Swarm/docker-compose 

Docker now has its own orchestrator: Swarm (compatible with docker-compose files). You have to enable it (the Mac version will also work with Linux).

Popis Stacku 

Now that you have Swarm working, create the service file (see Deploy to Swarm) qgis-stack.yaml:

version: '3.7'

services:
  qgis-server:
    # Should use version with utf-8 locale support:
    image: qgis-server:latest
    volumes:
    - REPLACE_WITH_FULL_PATH/data:/data:ro
    environment:
    - LANG=en_EN.UTF-8
    - QGIS_PROJECT_FILE=/data/osm.qgs
    - QGIS_SERVER_LOG_LEVEL=0  # INFO (log all requests)
    - DEBUG=1                  # display env before spawning QGIS Server

  nginx:
    image: nginx:1.13
    ports:
    - 8080:80
    volumes:
    - REPLACE_WITH_FULL_PATH/nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
    - qgis-server

Chcete-li nasadit (nebo aktualizovat) Stack, zadejte:

docker stack deploy -c qgis-stack.yaml qgis-stack

Check the stack deployment status until you obtain 1/1 in the replicas column:

docker stack services qgis-stack

Something like:

ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
gmx7ewlvwsqt        qgis_nginx          replicated          1/1                 nginx:1.13          *:8080->80/tcp
l0v2e7cl43u3        qgis_qgis-server    replicated          1/1                 qgis-server:latest

To check WMS capabilities, type in a web browser http://localhost:8080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

Cleanup 

Chcete-li vyčistit, zadejte:

docker stack rm qgis-stack

7.2.2. Kubernetes 

Instalace 

If you have a Docker Desktop installation, using Kubernetes (aka k8s) is pretty straight forward: enable k8s.

If not, follow the minikube tutorial or microk8s for Ubuntu.

As Kubernetes installation can be really complex, we will only focus on aspects used by this demo. For further / deeper information, check the official documentation.

microk8s

microk8s needs extra steps: you have to enable the registry and tag the qgis-server image in order to have Kubernetes to find the created images.

Nejprve povolte registr:

microk8s enable dashboard dns registry

Poté označte a nahrajte image do nově vytvořeného registru:

docker tag qgis-server 127.0.0.1:32000/qgis-server && docker push 127.0.0.1:32000/qgis-server

Finally, add or complete the /etc/docker/daemon.json to have your registry 127.0.0.1:32000 listed in the insecure-registries field:

{
  "insecure-registries": ["127.0.0.1:32000"]
}

Creating manifests 

Kubernetes popisuje objekty, které se mají nasadit do manifestů yaml. Existuje mnoho různých druhů, ale k vystavení nasazení interním nebo externím účelům použijeme pouze nasazení (manipulační pody, tj. Image dockeru) a služby.

Deployment manifests

Create a file deployments.yaml with this content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: qgis-server
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      myLabel: qgis-server
  template:
    metadata:
      labels:
        myLabel: qgis-server
    spec:
      containers:
        - name: qgis-server
          image: localhost:32000/qgis-server:latest
          imagePullPolicy: IfNotPresent
          env:
            - name: LANG
              value: en_EN.UTF-8
            - name: QGIS_PROJECT_FILE
              value: /data/osm.qgs
            - name: QGIS_SERVER_LOG_LEVEL
              value: "0"
            - name: DEBUG
              value: "1"
          ports:
            - containerPort: 5555
          volumeMounts:
            - name: qgis-data
              mountPath: /data/
      volumes:
        - name: qgis-data
          hostPath:
            path: REPLACE_WITH_FULL_PATH/data

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: qgis-nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      myLabel: qgis-nginx
  template:
    metadata:
      labels:
        myLabel: qgis-nginx
    spec:
      containers:
        - name: qgis-nginx
          image: nginx:1.13
          ports:
            - containerPort: 80
          volumeMounts:
            - name: nginx-conf
              mountPath: /etc/nginx/conf.d/default.conf
      volumes:
        - name: nginx-conf
          hostPath:
            path: REPLACE_WITH_FULL_PATH/nginx.conf

Service manifests

Create a file services.yaml with this content:

apiVersion: v1
kind: Service
metadata:
  name: qgis-server
  namespace: default
spec:
  type: ClusterIP
  selector:
    myLabel: qgis-server
  ports:
    - port: 5555
      targetPort: 5555
---
apiVersion: v1
kind: Service
metadata:
  name: qgis-nginx
  namespace: default
spec:
  type: NodePort
  selector:
    myLabel: qgis-nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080

Deploying manifests 

To deploy the images and services in Kubernetes, one can use the dashboard (click on the + on the upper right) or the command line.

Poznámka

When using the command line with microk8s you will have to prefix each command with microk8s.

To deploy or update your manifests:

kubectl apply -k ./

Chcete-li zkontrolovat, co je aktuálně nasazeno:

kubectl get pods,services,deployment

Měli byste získat něco jako:

NAME                               READY   STATUS    RESTARTS   AGE
pod/qgis-nginx-54845ff6f6-8skp9    1/1     Running   0          27m
pod/qgis-server-75df8ddd89-c7t7s   1/1     Running   0          27m

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service/Kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP        5h51m
service/qgis-exec-server   ClusterIP   10.152.183.218   <none>        5555/TCP       35m
service/qgis-nginx         NodePort    10.152.183.234   <none>        80:30080/TCP   27m
service/qgis-server        ClusterIP   10.152.183.132   <none>        5555/TCP       27m

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/qgis-nginx    1/1     1            1           27m
deployment.apps/qgis-server   1/1     1            1           27m

To read nginx/qgis logs, type:

kubectl logs -f POD_NAME

To check WMS capabilities, type in a web browser http://localhost:30080/qgis-server/?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

Cleanup 

Chcete-li vyčistit, zadejte:

kubectl delete -n default service/qgis-server service/qgis-nginx deployment/qgis-nginx deployment/qgis-server

7.3. Cloud deployment 

Správa vlastního clusteru serverů za účelem nasazení kontejnerových aplikací je složitá práce. Musíte zvládnout více problémů, jako je hardware, šířka pásma a zabezpečení na různých úrovních.

Cloudová řešení pro nasazení mohou být dobrou alternativou, když se nechcete soustředit na správu infrastruktury.

A cloud deployment may use proprietary mechanisms, but they are also compatible with the stages explained previously (docker images and stack management).

7.3.1. AWS usecase 

With Amazon AWS, through ECS (Elastic Container Service) functionalities, you can use docker-compose or Kubernetes compatible wrappers to manage your stack. You will have to create an image registry for your custom images to be accessible.

To use docker-compose alike functionalities, you need to install the ecs-cli client and have proper permissions / roles. Then, with the help of the ecs-cli compose commands (see the ecs-cli compose manual and ecs-cli tutorial), you can reuse the stack description.

To use Kubernetes, you can use the AWS web console or the command line tool eksctl and have the proper permissions / roles. Then with a well configured kubectl environment, you can reuse the Kubernetes manifests.