Following the first part, we are going to setup the production environment of our server through Docker Compose, Nginx setup and secure with SSL certificates

The next step is to setup the server to a production standard, for this we need a few components and docker compose which will tie everything together. The migrations we ran before were done so using Django's default database, SQlite, which is not usable in production when running a website with many datasets. PostreSQL is the principal database system we use when developping in Django. When it comes to show the website and its files, we will set up Gunicorn which will serve the website in production mode while NGINX will be used to serve the files to the outside. Finally, Certbot will make our website SSL-friendly, accessible through HTTPS protocol.

You can find the full project on GitLab so that when you follow this part, you can better assess where you are placing your files.

Installing NGINX

First let's setup NGINX by creating a directory inside my_project/:

mkdir my_project/nginx

Then making a nginx file:

nano my_project/nginx/default.conf.template

And paste the following configuration that contains both HTTP as well as secure HTTPS certified by Let's Encrypt with Certbot at the end of this tutorial :

upstream backend {
    server backend:8000;
}

server {

    listen 80;
    listen [::]:80;

    server_name ${DOMAIN} ${DOMAINWWW};


    return 301 https://$host$request_uri;

    ##buffer policy
    client_body_buffer_size 8M;
    client_header_buffer_size 1M;
    client_max_body_size 8M;
    large_client_header_buffers 2 1M;
    ##end buffer policy

    location / {
        proxy_pass http://backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }


    location /static/ {
    client_body_buffer_size 150M;
    client_max_body_size 150M;
        alias /app/static/;
    }

    location /media/ {
    client_body_buffer_size 150M;
    client_max_body_size 150M;
        alias /app/media/;
    }

}

In the previous Nginx default configuration file, we are using Nginx's newly deisgned variables syntax to use our environent variables as such: ${my_variable}.

Create a Dockerfile for nginx in the same directory:

nano Dockerfile

Paste the following:

FROM nginx:1.19.6-alpine

EXPOSE 80

RUN mkdir /etc/nginx/templates
COPY default.conf.template /etc/nginx/templates

RUN rm /etc/nginx/conf.d/default.conf

We are going to create a docker image just for nginx:

cd my_project/nginx
docker build . -t my_project/backend-ingress

In order for NGINX to work with Wagtail we need Docker to install psycopg2 alongside with everything inside our backend image. Edit the requirements.txt file:

nano my_project/my_project/requirements.txt

And add:

psycopg2
cd my_project/my_project

Production setup

Create the wait-for-it file which will check that the database is up and running before launching our wagtail server:

nano wait-for-it.sh

And paste:

#!/usr/bin/env bash
# Use this script to test if a given TCP host/port are available

WAITFORIT_cmdname=${0##*/}

echoerr() { if [[ $WAITFORIT_QUIET -ne 1 ]]; then echo "$@" 1>&2; fi }

usage()
{
    cat << USAGE >&2
Usage:
    $WAITFORIT_cmdname host:port [-s] [-t timeout] [-- command args]
    -h HOST | --host=HOST       Host or IP under test
    -p PORT | --port=PORT       TCP port under test
                                Alternatively, you specify the host and port as host:port
    -s | --strict               Only execute subcommand if the test succeeds
    -q | --quiet                Don't output any status messages
    -t TIMEOUT | --timeout=TIMEOUT
                                Timeout in seconds, zero for no timeout
    -- COMMAND ARGS             Execute command with args after the test finishes
USAGE
    exit 1
}

wait_for()
{
    if [[ $WAITFORIT_TIMEOUT -gt 0 ]]; then
        echoerr "$WAITFORIT_cmdname: waiting $WAITFORIT_TIMEOUT seconds for $WAITFORIT_HOST:$WAITFORIT_PORT"
    else
        echoerr "$WAITFORIT_cmdname: waiting for $WAITFORIT_HOST:$WAITFORIT_PORT without a timeout"
    fi
    WAITFORIT_start_ts=$(date +%s)
    while :
    do
        if [[ $WAITFORIT_ISBUSY -eq 1 ]]; then
            nc -z $WAITFORIT_HOST $WAITFORIT_PORT
            WAITFORIT_result=$?
        else
            (echo -n > /dev/tcp/$WAITFORIT_HOST/$WAITFORIT_PORT) >/dev/null 2>&1
            WAITFORIT_result=$?
        fi
        if [[ $WAITFORIT_result -eq 0 ]]; then
            WAITFORIT_end_ts=$(date +%s)
            echoerr "$WAITFORIT_cmdname: $WAITFORIT_HOST:$WAITFORIT_PORT is available after $((WAITFORIT_end_ts - WAITFORIT_start_ts)) seconds"
            break
        fi
        sleep 1
    done
    return $WAITFORIT_result
}

wait_for_wrapper()
{
    # In order to support SIGINT during timeout: http://unix.stackexchange.com/a/57692
    if [[ $WAITFORIT_QUIET -eq 1 ]]; then
        timeout $WAITFORIT_BUSYTIMEFLAG $WAITFORIT_TIMEOUT $0 --quiet --child --host=$WAITFORIT_HOST --port=$WAITFORIT_PORT --timeout=$WAITFORIT_TIMEOUT &
    else
        timeout $WAITFORIT_BUSYTIMEFLAG $WAITFORIT_TIMEOUT $0 --child --host=$WAITFORIT_HOST --port=$WAITFORIT_PORT --timeout=$WAITFORIT_TIMEOUT &
    fi
    WAITFORIT_PID=$!
    trap "kill -INT -$WAITFORIT_PID" INT
    wait $WAITFORIT_PID
    WAITFORIT_RESULT=$?
    if [[ $WAITFORIT_RESULT -ne 0 ]]; then
        echoerr "$WAITFORIT_cmdname: timeout occurred after waiting $WAITFORIT_TIMEOUT seconds for $WAITFORIT_HOST:$WAITFORIT_PORT"
    fi
    return $WAITFORIT_RESULT
}

# process arguments
while [[ $# -gt 0 ]]
do
    case "$1" in
        *:* )
        WAITFORIT_hostport=(${1//:/ })
        WAITFORIT_HOST=${WAITFORIT_hostport[0]}
        WAITFORIT_PORT=${WAITFORIT_hostport[1]}
        shift 1
        ;;
        --child)
        WAITFORIT_CHILD=1
        shift 1
        ;;
        -q | --quiet)
        WAITFORIT_QUIET=1
        shift 1
        ;;
        -s | --strict)
        WAITFORIT_STRICT=1
        shift 1
        ;;
        -h)
        WAITFORIT_HOST="$2"
        if [[ $WAITFORIT_HOST == "" ]]; then break; fi
        shift 2
        ;;
        --host=*)
        WAITFORIT_HOST="${1#*=}"
        shift 1
        ;;
        -p)
        WAITFORIT_PORT="$2"
        if [[ $WAITFORIT_PORT == "" ]]; then break; fi
        shift 2
        ;;
        --port=*)
        WAITFORIT_PORT="${1#*=}"
        shift 1
        ;;
        -t)
        WAITFORIT_TIMEOUT="$2"
        if [[ $WAITFORIT_TIMEOUT == "" ]]; then break; fi
        shift 2
        ;;
        --timeout=*)
        WAITFORIT_TIMEOUT="${1#*=}"
        shift 1
        ;;
        --)
        shift
        WAITFORIT_CLI=("$@")
        break
        ;;
        --help)
        usage
        ;;
        *)
        echoerr "Unknown argument: $1"
        usage
        ;;
    esac
done

if [[ "$WAITFORIT_HOST" == "" || "$WAITFORIT_PORT" == "" ]]; then
    echoerr "Error: you need to provide a host and port to test."
    usage
fi

WAITFORIT_TIMEOUT=${WAITFORIT_TIMEOUT:-15}
WAITFORIT_STRICT=${WAITFORIT_STRICT:-0}
WAITFORIT_CHILD=${WAITFORIT_CHILD:-0}
WAITFORIT_QUIET=${WAITFORIT_QUIET:-0}

# Check to see if timeout is from busybox?
WAITFORIT_TIMEOUT_PATH=$(type -p timeout)
WAITFORIT_TIMEOUT_PATH=$(realpath $WAITFORIT_TIMEOUT_PATH 2>/dev/null || readlink -f $WAITFORIT_TIMEOUT_PATH)

WAITFORIT_BUSYTIMEFLAG=""
if [[ $WAITFORIT_TIMEOUT_PATH =~ "busybox" ]]; then
    WAITFORIT_ISBUSY=1
    # Check if busybox timeout uses -t flag
    # (recent Alpine versions don't support -t anymore)
    if timeout &>/dev/stdout | grep -q -e '-t '; then
        WAITFORIT_BUSYTIMEFLAG="-t"
    fi
else
    WAITFORIT_ISBUSY=0
fi

if [[ $WAITFORIT_CHILD -gt 0 ]]; then
    wait_for
    WAITFORIT_RESULT=$?
    exit $WAITFORIT_RESULT
else
    if [[ $WAITFORIT_TIMEOUT -gt 0 ]]; then
        wait_for_wrapper
        WAITFORIT_RESULT=$?
    else
        wait_for
        WAITFORIT_RESULT=$?
    fi
fi

if [[ $WAITFORIT_CLI != "" ]]; then
    if [[ $WAITFORIT_RESULT -ne 0 && $WAITFORIT_STRICT -eq 1 ]]; then
        echoerr "$WAITFORIT_cmdname: strict mode, refusing to execute subprocess"
        exit $WAITFORIT_RESULT
    fi
    exec "${WAITFORIT_CLI[@]}"
else
    exit $WAITFORIT_RESULT
fi

Make the file an executable:

chmod +x wait-for-it.sh

Edit the next two files, they will be used in production by replacing their development counterpart.

nano my_project/manage.production.py

Place the following code inside:

#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "engineerx.settings.production")

    from django.core.management import execute_from_command_line

    execute_from_command_line(sys.argv)

Same goes for my_project/my_project/wsqi.production.py, create it and paste the code below:

nano my_project/my_project/wsqi.production.py

"""
WSGI config for my_project project.

It exposes the WSGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/3.1/howto/deployment/wsgi/
"""

import os

from django.core.wsgi import get_wsgi_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "my_project.settings.production")

application = get_wsgi_application()

And finally production.py:

nano my_project/my_project/my_project/settings/production.py
from .base import *

DEBUG = False

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.environ['SECRET_KEY']

# SECURITY WARNING: define the correct hosts in production!
ALLOWED_HOSTS = [os.environ['ALLOWED_HOST']]

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': os.environ['POSTGRES_PASSWORD'],
        'HOST': 'db',
        'PORT': '5432',
    }
}

try:
    from .local import *
except ImportError:
    pass

Add a bash script to start Gunicorn and replace the development files manage.py and wsgi.py by the production-ready files: nano my_project/my_project/start.sh

FILE=manage.production.py
if test -f "$FILE"; then
echo 'Start removed manage added prod'
rm manage.py && mv manage.production.py manage.py
rm my_project/wsgi.py && mv my_project/wsgi.production.py my_project/wsgi.py
fi

python manage.py makemigrations --noinput
python manage.py migrate --noinput
python manage.py collectstatic --noinput

gunicorn my_project.wsgi:application

As before, make it executable:

chmod +x start.sh

We now tie everything together using docker-compose by making a yaml file:

nano my_project/docker-compose.yaml
version: "3"
services:
  ingress:
    image: my_project/backend-ingress
    ports: 
      - "80:80"
    networks:
      - my_project-backend
    volumes: 
      - static:/home/app/web/static
      - media:/home/app/web/media
    depends_on:
      - backend
  backend:
    image: my_project/backend
    ports:
      - "8001:8000"
    entrypoint: ["/bin/sh","-c"]
    command:
    - |
       ./wait-for-it.sh db:5432 -- ./start.sh 
    environment:
      - POSTGRES_PASSWORD
      - SECRET_KEY
      - ALLOWED_HOST
    networks:
      - my_project-backend
    volumes: 
      - static:/app/static
      - media:/app/media
    depends_on:
      - db 
  db:
    image: postgres:latest
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_PASSWORD
    networks:
      - my_project-backend 
networks:
  my_project-backend:
    driver: bridge
volumes:
    static:
      driver: local
    media:
      driver: local

Now is the time to set the variables used in our production environment:

cd my_project
nano .env

Write:

POSTGRES_PASSWORD=the_password
POSTGRES_PASSWORD=the_password
SECRET_KEY=the_password
ALLOWED_HOST=the_ip
DOMAIN=your_domain.name
DOMAINWWW=www.your_domain.name
WAGTAILADMIN_BASE_URL="http://your_ip:8000" # or "http://your_domain.name:8000"

If you use my GitLab repository, don't forget to rename the env_variables to .env

To produce Django's secret key, you can use Djecrety to create a new secret key.

Same as before, logout of your ssh session and log back in or restart your machine.

Let docker run the server

docker compose up

Create the super user for the production side of your project from another SSH session:

docker compose exec backend python manage.py createsuperuser

To come back to your development environment, you need to do the migrations again:

docker run -it -p 8000:8000 my_project/backend bash
python manage.py makemigrations && python manage.py migrate
python manage.py runserver 0.0.0.0:8000

Change the ports suiting the development needs.

From then on, the workflow is as follows:

Develop your site using docker run my_project/backend, installing packages as needed using PiP. When your Wagtail app is ready for testing, you need to update the requirements file for Dockerfile. Inside my_project/my_prroject/ enter the command:

pip freeze > requirements.txt

Then we need to update the docker build. Still inside my_project/my_project, do:

docker build . -t backend

Go to the project's root directory and run:

docker compose up

Add a restart: always option under each services of your docker compose YAML file.

Making our website accessible through HTTPS

We just need to add our SSL certification and configure Nginx accordingly for the website to be able to display web pages in HTTPS. Edit docker-compose.yaml and add the next bit of code below the Nginx service as its own service:

# ...first part of the YAML file...
# add certbot as a service
  certbot:
    restart: always
    image: certbot/certbot:latest
    volumes:
      - ./certbot/www/:/var/www/certbot/:rw
      - ./certbot/conf/:/etc/letsencrypt/:rw
      # the next line ensures the certificate is renewed on expiration date
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
# ...rest of the code...

Then run:

docker compose run --rm  certbot certonly --webroot --webroot-path /var/www/certbot/ --dry-run -d your_domain.name

docker compose run --rm  certbot certonly --webroot --webroot-path /var/www/certbot/ -d your_domain.name

Edit our default Nginx file:

nano nginx/default.conf.template

Add the following at the end of the file:

# See here below our configuration to use HTTPS
server {
    listen 443 default_server ssl http2;
    listen [::]:443 ssl http2;

    server_name ${DOMAIN} ${DOMAINWWW};

        # This is to you tell Nginx 
        ssl_certificate /etc/nginx/ssl/live/${DOMAIN}/fullchain.pem;
        ssl_certificate_key /etc/nginx/ssl/live/${DOMAIN}/privkey.pem;


    ssl_protocols TLSv1.2 TLSv1.3;

    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA HIGH !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";

    ##buffer policy
    client_body_buffer_size 1K;
    client_header_buffer_size 1k;
    client_max_body_size 1k;
    large_client_header_buffers 2 1k;
    ##end buffer policy

    location / {
        proxy_pass http://backend;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location /static/ {
        alias /home/app/web/static/;
    }

    location /media/ {
         alias /home/app/web/media/;
    }


}

Edit the Dockerfile inside the nginx folder to add:

# ...first part of the config file...
EXPOSE 80
EXPOSE 443
# ...rest of the config file...

Edit docker-compose.yaml again and add another port to our Nginx image as well as new volumes to the Nginx service:

version: "3"
services:
  ingress:
    image: my_project/backend-ingress
    ports: 
      - "80:80"
        # This below is to open the ports to allow HTTPS connections 
        - "443:443"
# ...parts of the configuration for ingress service...
    volumes: 
      - static:/home/app/web/static
      - media:/home/app/web/media
      - ./certbot/www:/var/www/certbot/:ro
      - ./certbot/conf/:/etc/nginx/ssl/:ro

Rebuild the image:

docker build . -t my_project/backend-ingress

You are set and ready and are able to run your docker containers in the background:

docker compose up -d

In case you need to stop Docker Compose, run:

docker compose stop

wagtail-docker-nginx-part-2


Required for comment verification