Categories
SysOps

How to take advantage of Docker to install Mastodon

Take advantage of Docker to install Mastodon

This article is based on the official docker compose which can be found the in mastodon project repository.

Operating system.

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04 LTS
Release:	22.04
Codename:	jammy

Install and configure firewall

Update package index.

$ sudo apt update

Upgrade packages.

$ sudo apt upgrade

Display network interfaces.

$ ip -br a
lo               UNKNOWN        127.0.0.1/8
enp0s8           UP             192.168.56.4/24 

Install a dynamically managed firewall with support for network zones.

$ sudo apt install firewalld

Service will be enabled and started automatically.

$ sudo systemctl status firewalld.service 
* firewalld.service - firewalld - dynamic firewall daemon
     Loaded: loaded (/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-05-02 02:20:04 UTC; 11s ago
       Docs: man:firewalld(1)
   Main PID: 12924 (firewalld)
      Tasks: 2 (limit: 4665)
     Memory: 23.0M
        CPU: 250ms
     CGroup: /system.slice/firewalld.service
             `-12924 /usr/bin/python3 /usr/sbin/firewalld --nofork --nopid

May 02 02:20:04 ubuntu-jammy systemd[1]: Starting firewalld - dynamic firewall daemon...
May 02 02:20:04 ubuntu-jammy systemd[1]: Started firewalld - dynamic firewall daemon.

Add an external interfaces to the public zone.

$ sudo firewall-cmd --add-interface enp0s8 --zone public
success

Display and verify used zones.

$ sudo firewall-cmd --get-active-zones 
public
  interfaces: enp0s8

Display and verify allowed services.

$ sudo firewall-cmd --list-services --zone=public 
dhcpv6-client ssh

Enable http and https services.

$ sudo firewall-cmd --add-service http --add-service https --zone public
success

Display and verify public zone.

$ sudo firewall-cmd --list-all --zone=public
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp0s8
  sources: 
  services: dhcpv6-client http https ssh
  ports: 
  protocols: 
  forward: yes
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

Ensure that docker interface is in a trusted zone.

$ sudo firewall-cmd --add-interface=docker0 --zone trusted
success

Display and verify trusted zone.

$ sudo firewall-cmd --list-all --zone=trusted
trusted (active)
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: docker0
  sources: 
  services: 
  ports: 
  protocols: 
  forward: yes
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 

Add masquerade, as application will contact instances.

$ sudo firewall-cmd --zone=public --add-masquerade

Display and verify public zone.

$ sudo firewall-cmd --list-all --zone=public 
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp0s8
  sources: 
  services: dhcpv6-client http https ssh
  ports: 
  protocols: 
  forward: yes
  masquerade: yes
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 

Make changes permanent.

$ sudo firewall-cmd --runtime-to-permanent
success

Set server hostname

Define server hostname.

$ sudo hostnamectl --static set-hostname example.org

Increase the limit of the mmap counts

The default limit of the mmap counts is definitely too low for Elasticsearch service.

$ sysctl vm.max_map_count
vm.max_map_count = 65530

Increase it to avoid running out of map areas.

$ echo "vm.max_map_count=262144" | sudo tee /etc/sysctl.d/90-max_map_count.conf
$ sudo sysctl --load /etc/sysctl.d/90-max_map_count.conf

Install Docker

Install docker.

$ sudo apt install docker.io docker-compose

It will try to start, but it doesn’t play well with recent firewalld using nftables.

$ systemctl status docker
x docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2022-05-02 02:14:40 UTC; 6s ago
TriggeredBy: x docker.socket
       Docs: https://docs.docker.com
    Process: 2452 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
   Main PID: 2452 (code=exited, status=1/FAILURE)
        CPU: 59ms
$ sudo iptables -L -v -n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (0 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0  

Ensure that you are part of the docker group.

$ sudo usermod -a -G docker $USER

Alter docker configuration to log messages to journald.

$ cat <<EOF | sudo tee /etc/docker/daemon.json
{
  "iptables": false,
  "log-driver": "journald"
}
EOF

The best solution is to reboot operating system.

$ sudo reboot

Service will start normally.

$ sudo systemctl status docker
* docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-05-02 02:20:48 UTC; 2min 27s ago
TriggeredBy: * docker.socket
       Docs: https://docs.docker.com
   Main PID: 981 (dockerd)
      Tasks: 8
     Memory: 97.8M
        CPU: 151ms
     CGroup: /system.slice/docker.service
             `-981 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.187805266Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/cont>
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.187891761Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.224420439Z" level=info msg="[graphdriver] using prior storage driver: overlay2"
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.227674497Z" level=info msg="Loading containers: start."
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.247689532Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. >
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.264692171Z" level=info msg="Loading containers: done."
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.311939073Z" level=info msg="Docker daemon" commit=20.10.12-0ubuntu4 graphdriver(s)=overlay2 version>
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.312291592Z" level=info msg="Daemon has completed initialization"
May 02 02:20:48 example.org systemd[1]: Started Docker Application Container Engine.
May 02 02:20:48 example.org dockerd[981]: time="2022-05-02T02:20:48.342171039Z" level=info msg="API listen on /run/docker.sock"
$ sudo iptables -L -v -n
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination 

Everything is as expected as we are using firewalld/nftables. Alternatively, you could alter firewalld configuration to use iptables.

Create application definition

Create application directory.

$ sudo mkdir /opt/mastodon/

Create directories related to database operations.

$ sudo mkdir -p /opt/mastodon/database/{postgresql,redis,elasticsearch}

Create directories related to web operations.

$ sudo mkdir -p /opt/mastodon/web/{public,system}

Fix permissions on web directories.

$ sudo chown 991:991 /opt/mastodon/web/{public,system}

Fix permissions on elasticsearch directory.

$ sudo chown 1000 /opt/mastodon/database/elasticsearch

Create a docker-compose file.

$ cat << EOF | sudo tee /opt/mastodon/docker-compose.yml
version: '3'

services:
  postgresql:
    image: postgres:14
    env_file: database.env
    restart: always
    shm_size: 256mb
    healthcheck:
      test: ['CMD', 'pg_isready', '-U', 'postgres']
    volumes:
      - postgresql:/var/lib/postgresql/data
    networks:
      - internal_network

#  pgbouncer:
#    image: edoburu/pgbouncer:1.12.0
#    env_file: database.env
#    depends_on:
#      - postgresql
#    healthcheck:
#      test: ['CMD', 'pg_isready', '-h', 'localhost']
#    networks:
#      - internal_network

  redis:
    image: redis:7
    restart: always
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - redis:/data
    networks:
      - internal_network

  redis-volatile:
    image: redis:7
    restart: always
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    networks:
      - internal_network

  elasticsearch:
    image: elasticsearch:7.17.3
    restart: always
    env_file: database.env
    environment:
      - cluster.name=elasticsearch-mastodon
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - ingest.geoip.downloader.enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test: ["CMD-SHELL", "nc -z elasticsearch 9200"]
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
    networks:
      - internal_network

  website:
    image: tootsuite/mastodon:v3.5.1
    env_file: 
      - application.env
      - database.env
    command: bash -c "bundle exec rails s -p 3000"
    restart: always    
    depends_on:
      - postgresql
#      - pgbouncer
      - redis
      - redis-volatile
      - elasticsearch
    ports:
      - '127.0.0.1:3000:3000'
    networks:
      - internal_network
      - external_network
    healthcheck:
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
    volumes:
      - uploads:/mastodon/public/system

  shell:
    image: tootsuite/mastodon:v3.5.1
    env_file: 
      - application.env
      - database.env
    command: /bin/bash 
    restart: "no"
    networks:
      - internal_network
      - external_network
    volumes:
      - uploads:/mastodon/public/system

  streaming:
    image: tootsuite/mastodon:v3.5.1
    env_file: 
      - application.env
      - database.env
    command: node ./streaming
    restart: always
    depends_on:
      - postgresql
#      - pgbouncer
      - redis
      - redis-volatile
      - elasticsearch
    ports:
      - '127.0.0.1:4000:4000'
    networks:
      - internal_network
      - external_network
    healthcheck:
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']

  sidekiq:
    image: tootsuite/mastodon:v3.5.1
    env_file: 
      - application.env
      - database.env
    command: bundle exec sidekiq
    restart: always
    depends_on:
      - postgresql
#      - pgbouncer
      - redis
      - redis-volatile
      - website
    networks:
      - internal_network
      - external_network
    healthcheck:
      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
    volumes:
      - uploads:/mastodon/public/system

networks:
  external_network:
  internal_network:
    internal: true

volumes:
  postgresql:
    driver_opts:
      type: none
      device: /opt/mastodon/database/postgresql
      o: bind    
  redis:
    driver_opts:
      type: none
      device: /opt/mastodon/database/redis
      o: bind    
  elasticsearch:
    driver_opts:
      type: none
      device: /opt/mastodon/database/elasticsearch
      o: bind    
  uploads:
    driver_opts:
      type: none
      device: /opt/mastodon/web/system
      o: bind    
EOF

Generate secrets

Initialize empty application configuration as you can generate secrets using docker container.

$ sudo touch /opt/mastodon/application.env
$ sudo touch /opt/mastodon/database.env

Generate SECRET_KEY_BASE and OTP_SECRET.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell bundle exec rake secret
Creating mastodon_shell_run ... done
0701fa876663c4e4111953c251b5794dba8e1aac8f120fac8b674b5142bee8802ade6eb91266dc1df82e23399dc77e27c1c2e6b7b3e8f0ba2bce9f65064a5791

Alternatively use openssl utility.

$ openssl rand -hex 64
0309a599e8ef767d9aa0860a57afca3287ef79e18d1ed2c72be833f4c2c3191825acda9b906be5e76ebc7accb21d7eabb49ba11ead5bfa051d5e54fc8ef7ea0

Generate VAPID_PRIVATE_KEY and VAPID_PRIVATE_KEY.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell bundle exec rake mastodon:webpush:generate_vapid_key 
Creating mastodon_shell_run ... done
VAPID_PRIVATE_KEY=kKz6fGDFsSEV-q3IZh3dMCmsi8YyLTh7NSCxrxBT4dU=
VAPID_PUBLIC_KEY=BMf1qY_YSybedtBsksFkSTkW_GcZ76BJcEJyHx9UO1Ig9NML_9_vdZwAt-vf8POV7e5R6J17MbUZ-wXbdU9NQ_k=

Alternatively use openssl utility.

$ openssl ecparam -name prime256v1 -genkey -noout -out vapid_private_key.pem
$ openssl ec -in vapid_private_key.pem -pubout -out vapid_public_key.pem
read EC key
writing EC key
$ cat vapid_private_key.pem 
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEINS+CzUMTQChTeRDSAv9ESg4WUsA4IDW+7ASWnioOF4FoAoGCCqGSM49
AwEHoUQDQgAEKKxVlTTf+ctC8OA2Sh9qY5AQtLd7O/NDIVevLoyEuLUojUXf8Szn
dela0hk6DSOxshPcPWK5AE64e/doIZuZUg==
-----END EC PRIVATE KEY-----
$ cat vapid_public_key.pem 
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEKKxVlTTf+ctC8OA2Sh9qY5AQtLd7
O/NDIVevLoyEuLUojUXf8Szndela0hk6DSOxshPcPWK5AE64e/doIZuZUg==
-----END PUBLIC KEY-----
$ echo -n VAPID_PRIVATE_KEY=;cat vapid_private_key.pem | sed -e "1 d" -e "$ d" | tr -d "\n"; echo
VAPID_PRIVATE_KEY=MHcCAQEEINS+CzUMTQChTeRDSAv9ESg4WUsA4IDW+7ASWnioOF4FoAoGCCqGSM49AwEHoUQDQgAEKKxVlTTf+ctC8OA2Sh9qY5AQtLd7O/NDIVevLoyEuLUojUXf8Szndela0hk6DSOxshPcPWK5AE64e/doIZuZUg==
$ echo -n VAPID_PUBLIC_KEY=;cat vapid_public_key.pem | sed -e "1 d" -e "$ d" | tr -d "\n"; echo
VAPID_PUBLIC_KEY=MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEKKxVlTTf+ctC8OA2Sh9qY5AQtLd7O/NDIVevLoyEuLUojUXf8Szndela0hk6DSOxshPcPWK5AE64e/doIZuZUg==

Store these secrets securely and remember that changing these will break push notifications.

I have used openssl utility to generate postgresql and elasticsearch passwords.

$ openssl rand -hex 15
96a0a9a3720e6718f105619a5cc6ce

Perform initial configuration

Create database configuration file.

$ cat << EOF | sudo tee /opt/mastodon/database.env
# postgresql configuration
POSTGRES_USER=mastodon
POSTGRES_DB=mastodon_production
POSTGRES_PASSWORD=O6lOD6nF2LbhhJs1e7QL

# pgbouncer configuration
#POOL_MODE=transaction
#ADMIN_USERS=postgres,mastodon
#DATABASE_URL="postgres://mastodon:O6lOD6nF2LbhhJs1e7QL@postgresql:5432/mastodon_production"

# elasticsearch
ES_JAVA_OPTS=-Xms512m -Xmx512m
ELASTIC_PASSWORD=gpwETw6U875pbhnPxbo4

# mastodon database configuration
#DB_HOST=pgbouncer
DB_HOST=postgresql
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=O6lOD6nF2LbhhJs1e7QL
DB_PORT=5432

REDIS_HOST=redis
REDIS_PORT=6379

CACHE_REDIS_HOST=redis-volatile
CACHE_REDIS_PORT=6379

ES_ENABLED=true
ES_HOST=elasticsearch
ES_PORT=9200
ES_USER=elastic
ES_PASS=gpwETw6U875pbhnPxbo4
EOF

Create application configuration file.

$ cat << EOF | sudo tee /opt/mastodon/application.env
# environment
RAILS_ENV=production
NODE_ENV=production

# domain
LOCAL_DOMAIN=example.org

# redirect to the first profile
SINGLE_USER_MODE=true

# do not serve static files
RAILS_SERVE_STATIC_FILES=false

# concurrency
WEB_CONCURRENCY=2
MAX_THREADS=5

# pgbouncer
#PREPARED_STATEMENTS=false

# locale
DEFAULT_LOCALE=en

# email, not used
SMTP_SERVER=localhost
SMTP_PORT=587
SMTP_FROM_ADDRESS=notifications@example.org

# secrets
SECRET_KEY_BASE=4fab322678d6baccaacc70a83937c883fb1ca7a30cafc826970e03b011b77d29377ae2c87b613dc234c083ccc63390847e9029aa9e2baf434e85352cbede616d
OTP_SECRET=58b2814d5692263875d072d1a6d873fc822d1be034ebd2402850bf2915c36d46d3ed35f00d6fa9a102f5d7e30586d6a163414eb8d0d72f8393adc9667f9c8af9

VAPID_PRIVATE_KEY=kKz6fGDFsSEV-q3IZh3dMCmsi8YyLTh7NSCxrxBT4dU=
VAPID_PUBLIC_KEY=BMf1qY_YSybedtBsksFkSTkW_GcZ76BJcEJyHx9UO1Ig9NML_9_vdZwAt-vf8POV7e5R6J17MbUZ-wXbdU9NQ_k=
EOF

Secure these files.

$ sudo chmod 600 /opt/mastodon/application.env 
$ sudo chmod 600 /opt/mastodon/database.env 

Extract static files

Create a temporary volume pointing to /opt/mastodon/web/public directory.

$ sudo docker volume create --opt type=none --opt device=/opt/mastodon/web/public --opt o=bind temporary_static

Copy static files.

$ sudo docker run --rm -v "temporary_static:/static" tootsuite/mastodon:v3.5.1 bash -c "cp -r /opt/mastodon/public/* /static/"

Remove temporary volume.

$ sudo docker volume rm temporary_static

That way we can serve these files using nginx.

$ ls /opt/mastodon/web/public/
500.html                    avatars            emoji            inert.css           oops.gif    shortcuts                 web-push-icon_favourite.png
android-chrome-192x192.png  badge.png          favicon-dev.ico  mask-icon.svg       oops.png    sounds                    web-push-icon_reblog.png
apple-touch-icon.png        browserconfig.xml  favicon.ico      mstile-150x150.png  packs       sw.js
assets                      embed.js           headers          ocr                 robots.txt  web-push-icon_expand.png

Delete and copy static files after every mastodon container upgrade.

Install web-server

Install nginx web-server.

$ sudo apt install nginx

Disable default virtual host.

$ sudo unlink /etc/nginx/sites-enabled/default

Create a directory for SSL certificates.

$ sudo mkdir /etc/nginx/ssl

Copy or create a example.org.crt certificate and example.org.key private key.

$ sudo openssl req -subj "/commonName=example.org/" -x509 -nodes -days 730 -newkey rsa:2048 -keyout /etc/nginx/ssl/example.org.key -out /etc/nginx/ssl/example.org.crt

Create a virtual host.

$ cat << 'EOF' | sudo tee /etc/nginx/sites-available/mastodon
map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

upstream backend {
    server 127.0.0.1:3000 fail_timeout=0;
}

upstream streaming {
    server 127.0.0.1:4000 fail_timeout=0;
}

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE:10m inactive=7d max_size=1g;

server {
  listen 80;
  server_name example.org;
  location / { return 301 https://$host$request_uri; }
}

server {
  listen 443 ssl http2;
  server_name example.org;

  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;
  ssl_session_tickets off;

  ssl_certificate     /etc/nginx/ssl/example.org.crt;
  ssl_certificate_key /etc/nginx/ssl/example.org.key;

  keepalive_timeout    70;
  sendfile             on;
  client_max_body_size 80m;

  root /opt/mastodon/public;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml image/x-icon;

  add_header Strict-Transport-Security "max-age=31536000" always;

  location / {
    try_files $uri @proxy;
  }

  location ~ ^/(system/accounts/avatars|system/media_attachments/files) {
    add_header Cache-Control "public, max-age=31536000, immutable";
    add_header Strict-Transport-Security "max-age=31536000" always;
    root /opt/mastodon/;
    try_files $uri @proxy;
  }

  location ~ ^/(emoji|packs) {
    add_header Cache-Control "public, max-age=31536000, immutable";
    add_header Strict-Transport-Security "max-age=31536000" always;
    try_files $uri @proxy;
  }

  location /sw.js {
    add_header Cache-Control "public, max-age=0";
    add_header Strict-Transport-Security "max-age=31536000" always;
    try_files $uri @proxy;
  }

  location @proxy {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Proxy "";
    proxy_pass_header Server;

    proxy_pass http://backend;
    proxy_buffering on;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    proxy_cache CACHE;
    proxy_cache_valid 200 7d;
    proxy_cache_valid 410 24h;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    add_header X-Cached $upstream_cache_status;
    add_header Strict-Transport-Security "max-age=31536000" always;

    tcp_nodelay on;
  }

  location /api/v1/streaming {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Proxy "";

    proxy_pass http://streaming;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;
  }

  error_page 500 501 502 503 504 /500.html;
}
EOF

Enable virtual host.

$ sudo ln -s /etc/nginx/sites-available/mastodon /etc/nginx/sites-enabled/

Restart web server.

$ sudo systemctl restart nginx

Preferably you should ensure that nginx is stopped and start it after adding initial user and disabling registrations if that is what you want.

Start mastodon service

Pull images first.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml pull
Pulling postgresql     ... done
Pulling redis          ... done
Pulling redis-volatile ... done
Pulling elasticsearch  ... done
Pulling website        ... done
Pulling shell          ... done
Pulling streaming      ... done
Pulling sidekiq        ... done

Create mastodon service file.

$ cat << EOF | sudo tee /etc/systemd/system/mastodon.service
[Unit]
Description=Mastodon service
After=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes

WorkingDirectory=/opt/mastodon
ExecStart=/usr/bin/docker-compose -f /opt/mastodon/docker-compose.yml up -d
ExecStop=/usr/bin/docker-compose -f /opt/mastodon/docker-compose.yml down

[Install]
WantedBy=multi-user.target
EOF
$ sudo systemctl daemon-reload

Start postgresql database and pgbouncer.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml up -d postgresql redis redis-volatile
Creating mastodon_postgresql_1     ... done
Creating mastodon_redis_1          ... done
Creating mastodon_redis-volatile_1 ... done

Wait till database starts.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml ps
          Name                         Command                  State       Ports
---------------------------------------------------------------------------------
mastodon_postgresql_1       docker-entrypoint.sh postgres    Up (healthy)        
mastodon_redis-volatile_1   docker-entrypoint.sh redis ...   Up (healthy)        
mastodon_redis_1            docker-entrypoint.sh redis ...   Up (healthy)

Setup database using shell container.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell bundle exec rake db:setup

Next time, after upgrade execute database migrations.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell bundle exec rake db:migrate

Start and enable service.

$ sudo systemctl enable --now mastodon.service

It should be all green after a while.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml ps
          Name                         Command                  State                     Ports               
--------------------------------------------------------------------------------------------------------------
mastodon_elasticsearch_1    /bin/tini -- /usr/local/bi ...   Up (healthy)                                     
mastodon_postgresql_1       docker-entrypoint.sh postgres    Up (healthy)                                     
mastodon_redis-volatile_1   docker-entrypoint.sh redis ...   Up (healthy)                                     
mastodon_redis_1            docker-entrypoint.sh redis ...   Up (healthy)                                     
mastodon_shell_1            /usr/bin/tini -- /bin/bash       Exit 0                                           
mastodon_sidekiq_1          /usr/bin/tini -- bundle ex ...   Up (healthy)   3000/tcp, 4000/tcp                
mastodon_streaming_1        /usr/bin/tini -- node ./st ...   Up (healthy)   3000/tcp, 127.0.0.1:4000->4000/tcp
mastodon_website_1          /usr/bin/tini -- bash -c b ...   Up (healthy)   127.0.0.1:3000->3000/tcp, 4000/tcp

Create user.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell bin/tootctl accounts create milosz --email nonexistingmilosz@google.com --confirmed --role admin
Creating mastodon_shell_run ... done
OK
New password: a2af76138ca9502b1d1e1956aa8727e8

Disable registrations.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell  bin/tootctl settings registrations close
OK
Creating mastodon_shell_run ... done
OK
New password: 9b87fdef5b46b165aa00e24faaa6252c

Create, upgrade and populate Elasticsearch indices.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell tootctl search deploy
Creating mastodon_shell_run ... done
2/2 |=======================================================================| Time: 00:00:01 (2 docs/s)
Indexed 2 records, de-indexed 0

Create cleanup jobs

Remove downloaded media files.

$ cat << EOF | sudo tee /etc/systemd/system/mastodon-media-remove.service
[Unit]
Description=Mastodon - media remove service
Wants=mastodon-media-remove.timer

[Service]
Type=oneshot
StandardError=null
StandardOutput=null

WorkingDirectory=/opt/mastodon
ExecStart=/usr/bin/docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell tootctl media remove

[Install]
WantedBy=multi-user.target
EOF
$ cat << EOF | sudo tee /etc/systemd/system/mastodon-media-remove.timer
[Unit]
Description=Schedule a media remove every week

[Timer]
Persistent=true
OnCalendar=Sat *-*-* 00:00:00
Unit=mastodon-media-remove.service

[Install]
WantedBy=timers.target
EOF

Remove preview cards.

$ cat << EOF | sudo tee /etc/systemd/system/mastodon-preview_cards-remove.service
[Unit]
Description=Mastodon - preview cards remove service
Wants=mastodon-preview_cards-remove.timer

[Service]
Type=oneshot
StandardError=null
StandardOutput=null

WorkingDirectory=/opt/mastodon
ExecStart=/usr/bin/docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell tootctl preview_cards remove

[Install]
WantedBy=multi-user.target
EOF
$ cat << EOF | sudo tee /etc/systemd/system/mastodon-preview_cards-remove.timer
[Unit]
Description=Schedule a preview cards remove every week

[Timer]
Persistent=true
OnCalendar=Sat *-*-* 00:00:00
Unit=mastodon-preview_cards-remove.service

[Install]
WantedBy=timers.target
EOF

Reload systemd configuration.

$ sudo systemctl daemon-reload

Enable these timers (enable & start).

$ sudo systemctl enable --now mastodon-preview_cards-remove.timer
$ sudo systemctl enable --now mastodon-media-remove.timer

List timers.

$ systemctl list-timers      
NEXT                        LEFT              LAST                        PASSED       UNIT                                ACTIVATES                            
Sun 2022-05-01 23:23:06 UTC 55min left        Sun 2022-05-01 16:33:47 UTC 5h 54min ago ua-timer.timer                      ua-timer.service
Mon 2022-05-02 00:00:00 UTC 1h 31min left     Sun 2022-05-01 00:00:13 UTC 22h ago      dpkg-db-backup.timer                dpkg-db-backup.service
Mon 2022-05-02 00:00:00 UTC 1h 31min left     Sun 2022-05-01 00:00:13 UTC 22h ago      logrotate.timer                     logrotate.service
Mon 2022-05-02 00:46:08 UTC 2h 18min left     Thu 2022-04-28 19:30:47 UTC 3 days ago   fstrim.timer                        fstrim.service
Mon 2022-05-02 04:36:10 UTC 6h left           Sun 2022-05-01 10:35:05 UTC 11h ago      man-db.timer                        man-db.service
Mon 2022-05-02 06:06:01 UTC 7h left           Sun 2022-05-01 18:31:50 UTC 3h 56min ago fwupd-refresh.timer                 fwupd-refresh.service
Mon 2022-05-02 06:16:36 UTC 7h left           Sun 2022-05-01 16:17:01 UTC 6h ago       motd-news.timer                     motd-news.service
Mon 2022-05-02 06:57:01 UTC 8h left           Sun 2022-05-01 06:02:19 UTC 16h ago      apt-daily-upgrade.timer             apt-daily-upgrade.service
Mon 2022-05-02 08:34:26 UTC 10h left          Sun 2022-05-01 19:00:26 UTC 3h 27min ago apt-daily.timer                     apt-daily.service
Mon 2022-05-02 19:50:21 UTC 21h left          Sun 2022-05-01 19:50:21 UTC 2h 37min ago update-notifier-download.timer      update-notifier-download.service
Mon 2022-05-02 19:59:15 UTC 21h left          Sun 2022-05-01 19:59:15 UTC 2h 28min ago systemd-tmpfiles-clean.timer        systemd-tmpfiles-clean.service
Sat 2022-05-07 00:00:00 UTC 5 days left       n/a                         n/a          mastodon-media-remove.timer         mastodon-media-remove.service
Sat 2022-05-07 00:00:00 UTC 5 days left       n/a                         n/a          mastodon-preview_cards-remove.timer mastodon-preview_cards-remove.service
Sun 2022-05-08 03:10:12 UTC 6 days left       Sun 2022-05-01 03:10:14 UTC 19h ago      e2scrub_all.timer                   e2scrub_all.service
Tue 2022-05-10 14:22:45 UTC 1 week 1 day left Sun 2022-05-01 22:26:16 UTC 1min 45s ago update-notifier-motd.timer          update-notifier-motd.service

Additional notes

Use tootctl utility for basic maintenance tasks.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell tootctl
Creating mastodon_shell_run ... done
Commands:
  tootctl accounts SUBCOMMAND ...ARGS                # Manage accounts
  tootctl cache SUBCOMMAND ...ARGS                   # Manage cache
  tootctl canonical_email_blocks SUBCOMMAND ...ARGS  # Manage canonical e-mail blocks
  tootctl domains SUBCOMMAND ...ARGS                 # Manage account domains
  tootctl email_domain_blocks SUBCOMMAND ...ARGS     # Manage e-mail domain blocks
  tootctl emoji SUBCOMMAND ...ARGS                   # Manage custom emoji
  tootctl feeds SUBCOMMAND ...ARGS                   # Manage feeds
  tootctl help [COMMAND]                             # Describe available commands or one specific command
  tootctl ip_blocks SUBCOMMAND ...ARGS               # Manage IP blocks
  tootctl maintenance SUBCOMMAND ...ARGS             # Various maintenance utilities
  tootctl media SUBCOMMAND ...ARGS                   # Manage media files
  tootctl preview_cards SUBCOMMAND ...ARGS           # Manage preview cards
  tootctl search SUBCOMMAND ...ARGS                  # Manage the search engine
  tootctl self-destruct                              # Erase the server from the federation
  tootctl settings SUBCOMMAND ...ARGS                # Manage dynamic settings
  tootctl statuses SUBCOMMAND ...ARGS                # Manage statuses
  tootctl upgrade SUBCOMMAND ...ARGS                 # Various version upgrade utilities
  tootctl version                                    # Show version

Reset password in case of emergency.

$ sudo docker-compose -f /opt/mastodon/docker-compose.yml run --rm shell tootctl accounts modify milosz --reset-password

Use acme.sh to issue Let's Encrypt certificate.

NTP is configured by default, but I have skipped parts related to enforcing key based secure shell authentication as this article is already too long.

$ timedatectl 
               Local time: Mon 2022-05-02 01:37:38 UTC
           Universal time: Mon 2022-05-02 01:37:38 UTC
                 RTC time: Mon 2022-05-02 01:37:37
                Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

Look at the OpenSSL::PKey::RSAError on search #18106 if on Mastodon 3.5.1 you get 503 HTTP error when searching for people after fresh install.

Follow up

Sending VAPID identified WebPush Notifications via Mozilla’s Push Service

Mastodon Is Like Twitter Without Nazis, So Why Are We Not Using It?

UNTESTED: Adding pgbouncer to docker-compose stack. #18208