CloudTada | Infrastructure & DevOps Insights

Installing Patroni

Objectives

After this lesson, you will:

Install Python and required dependencies
Install Patroni via pip
Understand the structure of patroni.yml file
Create systemd service for Patroni
Install Patroni on 3 nodes

1. Introduction

Patroni is a Python application, so it requires Python runtime and dependencies. In this lesson, we will:

Install Python 3 and pip
Install Patroni package
Configure Patroni
Create systemd service to manage Patroni daemon

Target architecture:

TEXT

┌──────────────────────────────────┐
│      etcd Cluster (3 nodes)      │
│         ✅ RUNNING                │
└──────────────────────────────────┘
         │        │        │
    ┌────┴────┐  │  ┌─────┴─────┐
    ▼         ▼  ▼  ▼           ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Patroni 1│ │Patroni 2│ │Patroni 3│ ← Installing now
│  + PG   │ │  + PG   │ │  + PG   │
└─────────┘ └─────────┘ └─────────┘

2. Installing Python Dependencies

2.1. Installing Python on Ubuntu/Debian

Perform on ALL 3 nodes.

Step 1: Install Python 3 and pip

TEXT

# Update package list
sudo apt update

# Install Python 3, pip, and development tools
sudo apt install -y python3 python3-pip python3-dev python3-venv

# Check version
python3 --version
# Output: Python 3.10.x (or 3.11.x or 3.12.x)

pip3 --version
# Output: pip 22.x.x

Step 2: Install system dependencies

TEXT

# Install libraries required for Patroni and PostgreSQL Python modules
sudo apt install -y \
  libpq-dev \
  gcc \
  python3-psycopg2

# libpq-dev: PostgreSQL client library headers
# gcc: Compiler for building Python packages
# python3-psycopg2: PostgreSQL adapter for Python

2.2. Installing Python on CentOS/RHEL

TEXT

# Python 3 is usually available, but need pip and dev tools
sudo dnf install -y python3 python3-pip python3-devel

# System dependencies
sudo dnf install -y \
  postgresql18-devel \
  gcc \
  python3-psycopg2

# Check
python3 --version
pip3 --version

2.3. Upgrade pip (recommended)

TEXT

# Upgrade pip to latest version
sudo pip3 install --upgrade pip

# Verify
pip3 --version

3. Installing Patroni via pip

3.1. Installing Patroni

Perform on ALL 3 nodes.

TEXT

# Install Patroni with etcd support
sudo pip3 install patroni[etcd]

# Or specify specific version
sudo pip3 install patroni[etcd]==3.2.0

# Check installation
patroni --version
# Output: patroni 3.2.0

patronictl --version
# Output: patronictl 3.2.0

Explanation of [etcd]:

Patroni supports multiple DCS backends (etcd, consul, zookeeper)
[etcd] installs additional python-etcd client library
Can install multiple backends: patroni[etcd,consul,zookeeper]

3.2. Installed packages

TEXT

# List related packages
pip3 list | grep -E "(patroni|etcd|psycopg)"

# Output:
# patroni          3.2.0
# python-etcd      0.4.5
# psycopg2         2.9.x
# psycopg2-binary  2.9.x

3.3. Verify Patroni commands

TEXT

# Check patroni binary
which patroni
# Output: /usr/local/bin/patroni

# Check patronictl binary
which patronictl
# Output: /usr/local/bin/patronictl

# List patronictl commands
patronictl --help

Output:

TEXT

Usage: patronictl [OPTIONS] COMMAND [ARGS]...

Commands:
  list        Show cluster members
  switchover  Perform planned switchover
  failover    Perform manual failover
  reinit      Reinitialize cluster member
  restart     Restart cluster member
  reload      Reload cluster member configuration
  pause       Disable auto-failover
  resume      Enable auto-failover
  edit-config Edit cluster configuration
  ...

4. Structure of patroni.yml file

4.1. Overview of patroni.yml

File patroni.yml is the main configuration file of Patroni, including:

Scope: Cluster name
Namespace: Prefix for keys in DCS
Node information: Node name, REST API config
DCS connection: etcd endpoints
Bootstrap: Initial cluster setup
PostgreSQL: Database configuration
Tags: Node metadata
Watchdog: Optional hardware watchdog

4.2. Basic structure

TEXT

scope: postgres
namespace: /service/
name: node1

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.11:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        max_wal_senders: 10
        max_replication_slots: 10

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 scram-sha-256

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.11:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

4.3. Explanation of main sections

Section: Global

TEXT

scope: postgres          # Cluster name (unique identifier)
namespace: /service/     # DCS key prefix
name: node1             # Unique node name in cluster

Section: REST API

TEXT

restapi:
  listen: 0.0.0.0:8008              # Listen on all interfaces
  connect_address: 10.0.1.11:8008   # Address for other nodes to connect
  # authentication:                  # Optional: Basic auth
  #   username: admin
  #   password: secret

REST API endpoints:

GET /: Cluster info
GET /health: Health check (200 = healthy)
GET /primary: Check if node is primary
GET /replica: Check if node is replica
POST /restart: Restart PostgreSQL
POST /reload: Reload configuration

Section: etcd (DCS)

TEXT

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379  # etcd endpoints
  # protocol: http      # http or https
  # username: user      # Optional: etcd authentication
  # password: pass

Section: Bootstrap

TEXT

bootstrap:
  dcs:
    ttl: 30                           # Leader lock TTL (seconds)
    loop_wait: 10                     # Check interval (seconds)
    retry_timeout: 10                 # DCS operation timeout
    maximum_lag_on_failover: 1048576  # Max lag (bytes) to be eligible for failover
    postgresql:
      use_pg_rewind: true             # Enable pg_rewind for fast recovery
      parameters:                     # PostgreSQL parameters
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"           # Required for pg_rewind

  initdb:                             # pg_initdb options
    - encoding: UTF8
    - data-checksums                  # Enable page checksums
    - locale: en_US.UTF-8

  pg_hba:                             # pg_hba.conf entries
    - host replication replicator 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 scram-sha-256

  users:                              # Create users during bootstrap
    admin:
      password: admin_password
      options:
        - createrole
        - createdb

Note: Bootstrap section only applies when initializing cluster for the first time.

Section: PostgreSQL

TEXT

postgresql:
  listen: 0.0.0.0:5432                    # PostgreSQL listen address
  connect_address: 10.0.1.11:5432         # Address to connect from outside
  data_dir: /var/lib/postgresql/18/data   # Data directory
  bin_dir: /usr/lib/postgresql/18/bin     # PostgreSQL binaries
  pgpass: /tmp/pgpass                     # Optional: pgpass file
  
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
    rewind:                                # Optional: dedicated user for pg_rewind
      username: rewind_user
      password: rewind_password
  
  parameters:                              # PostgreSQL runtime parameters
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9
    wal_buffers: 16MB
    default_statistics_target: 100
    random_page_cost: 1.1
    effective_io_concurrency: 200
    work_mem: 16MB
    min_wal_size: 1GB
    max_wal_size: 2GB
  
  pg_hba:                                  # Additional pg_hba.conf entries
    - host all all 10.0.2.0/24 scram-sha-256
  
  callbacks:                               # Optional: callback scripts
    on_start: /etc/patroni/scripts/on_start.sh
    on_stop: /etc/patroni/scripts/on_stop.sh
    on_role_change: /etc/patroni/scripts/on_role_change.sh

Section: Tags

TEXT

tags:
  nofailover: false      # false = can become primary
  noloadbalance: false   # false = can receive read traffic
  clonefrom: false       # false = can clone from this node
  nosync: false          # false = can become synchronous standby
  # Custom tags
  datacenter: dc1
  environment: production

Section: Watchdog (Optional)

TEXT

watchdog:
  mode: required         # required, automatic, or off
  device: /dev/watchdog  # Hardware watchdog device
  safety_margin: 5       # Seconds before watchdog triggers

5. Create Patroni configuration files

5.1. Create configuration directories

On ALL 3 nodes:

TEXT

# Create directory for Patroni config
sudo mkdir -p /etc/patroni

# Create directory for callback scripts (optional)
sudo mkdir -p /etc/patroni/scripts

# Set ownership
sudo chown -R postgres:postgres /etc/patroni

5.2. Node 1 - /etc/patroni/patroni.yml

TEXT

scope: postgres
namespace: /service/
name: node1

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.11:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.11:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.3. Node 2 - /etc/patroni/patroni.yml

TEXT

scope: postgres
namespace: /service/
name: node2

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.12:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.12:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.4. Node 3 - /etc/patroni/patroni.yml

TEXT

scope: postgres
namespace: /service/
name: node3

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.13:8008

etcd:
  hosts: 10.0.1.11:2379,10.0.1.12:2379,10.0.1.13:2379

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    postgresql:
      use_pg_rewind: true
      parameters:
        wal_level: replica
        hot_standby: "on"
        wal_keep_size: "1GB"
        max_wal_senders: 10
        max_replication_slots: 10
        wal_log_hints: "on"

  initdb:
    - encoding: UTF8
    - data-checksums

  pg_hba:
    - host replication replicator 10.0.1.11/32 scram-sha-256
    - host replication replicator 10.0.1.12/32 scram-sha-256
    - host replication replicator 10.0.1.13/32 scram-sha-256
    - host all all 10.0.1.0/24 scram-sha-256
    - host all all 0.0.0.0/0 md5

  users:
    admin:
      password: admin123
      options:
        - createrole
        - createdb

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.13:5432
  data_dir: /var/lib/postgresql/18/data
  bin_dir: /usr/lib/postgresql/18/bin
  authentication:
    replication:
      username: replicator
      password: replicator_password
    superuser:
      username: postgres
      password: postgres_password
  parameters:
    max_connections: 100
    shared_buffers: 2GB
    effective_cache_size: 6GB
    maintenance_work_mem: 512MB
    checkpoint_completion_target: 0.9

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: false
  nosync: false

5.5. Set permissions

On ALL 3 nodes:

TEXT

# Set ownership
sudo chown postgres:postgres /etc/patroni/patroni.yml

# Secure permissions (file contains passwords)
sudo chmod 600 /etc/patroni/patroni.yml

# Verify
ls -l /etc/patroni/patroni.yml
# Output: -rw------- 1 postgres postgres ... patroni.yml

6. Create systemd service for Patroni

6.1. Create systemd unit file

Create file /etc/systemd/system/patroni.service on ALL 3 nodes:

TEXT

[Unit]
Description=Patroni PostgreSQL HA manager
Documentation=https://patroni.readthedocs.io/
After=syslog.target network.target etcd.service

[Service]
Type=simple
User=postgres
Group=postgres

# Start Patroni
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml

# Reload configuration
ExecReload=/bin/kill -HUP $MAINPID

# Restart behavior
Restart=on-failure
RestartSec=5

# Limits
LimitNOFILE=65536
LimitNPROC=65536

# Logging
StandardOutput=journal
StandardError=journal

# Working directory
WorkingDirectory=/var/lib/postgresql

# Environment
Environment=PATH=/usr/lib/postgresql/18/bin:/usr/local/bin:/usr/bin:/bin

[Install]
WantedBy=multi-user.target

6.2. Explanation of systemd unit file

Directive	Explanation
After=etcd.service	Start after etcd is ready
Type=simple	Process runs in foreground
User=postgres	Run with postgres user
ExecStart	Command to start Patroni
ExecReload	Send HUP signal to reload config
Restart=on-failure	Auto restart if fails
RestartSec=5	Wait 5 seconds before restart
LimitNOFILE=65536	Max open files
StandardOutput=journal	Log to systemd journal

6.3. Enable and verify service

On ALL 3 nodes:

TEXT

# Reload systemd
sudo systemctl daemon-reload

# Enable Patroni service (auto-start on boot)
sudo systemctl enable patroni

# Verify service file
systemctl cat patroni

# Check status (should be inactive/dead - not started yet)
systemctl status patroni

7. Verify installation

7.1. Check Patroni installation

On ALL 3 nodes:

TEXT

# Check Patroni version
patroni --version

# Check patronictl
patronictl --help

# Verify config file
sudo -u postgres cat /etc/patroni/patroni.yml

# Validate YAML syntax
python3 -c "import yaml; yaml.safe_load(open('/etc/patroni/patroni.yml'))"
# No output = valid YAML

7.2. Check etcd connectivity

TEXT

# Test etcd from each node
etcdctl endpoint health --cluster

# Should see all 3 etcd nodes healthy

7.3. Pre-flight checklist

Before starting Patroni, verify:

TEXT

# ✅ PostgreSQL installed but NOT running
systemctl status postgresql
# Should be: inactive (dead)

# ✅ etcd cluster healthy
etcdctl endpoint health --cluster

# ✅ Patroni config file exists and has correct permissions
ls -l /etc/patroni/patroni.yml

# ✅ Data directory has correct ownership
ls -ld /var/lib/postgresql/18/data
# Owner: postgres:postgres

# ✅ systemd service enabled
systemctl is-enabled patroni
# Output: enabled

# ✅ Firewall rules (if firewall exists)
sudo ufw status | grep -E "(5432|8008)"
# Or
sudo firewall-cmd --list-ports | grep -E "(5432|8008)"

8. Troubleshooting

8.1. Issue: pip install fails

TEXT

# Error: "No module named 'setuptools'"
# Solution:
sudo apt install python3-setuptools

# Or upgrade pip
sudo pip3 install --upgrade pip setuptools

8.2. Issue: Cannot find PostgreSQL binaries

TEXT

# Error: "could not find postgres binary"
# Solution: Check bin_dir in patroni.yml

# Find PostgreSQL bin directory
which postgres
# Output: /usr/lib/postgresql/18/bin/postgres

# Update patroni.yml
postgresql:
  bin_dir: /usr/lib/postgresql/18/bin

8.3. Issue: Permission denied on data directory

TEXT

# Error: "FATAL:  data directory ... has wrong ownership"
# Solution:
sudo chown -R postgres:postgres /var/lib/postgresql/18/data
sudo chmod 700 /var/lib/postgresql/18/data

8.4. Issue: YAML syntax error

TEXT

# Validate YAML
python3 -c "import yaml; yaml.safe_load(open('/etc/patroni/patroni.yml'))"

# Common issues:
# - Mixed tabs and spaces (use spaces only)
# - Incorrect indentation
# - Missing quotes around special characters
# - Duplicate keys

9. Summary

Key Takeaways

✅ Patroni: Python application, installed via pip

✅ Dependencies: Python 3, pip, psycopg2, python-etcd

✅ Configuration: patroni.yml contains all settings

✅ systemd service: Manages Patroni daemon

✅ Security: Config file has permissions 600 (contains passwords)

Checklist after Lab

Python 3 and pip installed on all 3 nodes
Patroni 3.2.0+ installed on all 3 nodes
File /etc/patroni/patroni.yml created on each node with individual config
systemd service patroni.service created and enabled
Correct permissions for config file (600, owner postgres)
etcd cluster running and healthy

Current Architecture

TEXT

✅ 3 VMs prepared (Lesson 4)
✅ PostgreSQL 18 installed (Lesson 5)
✅ etcd cluster running (Lesson 6)
✅ Patroni installed and configured (Lesson 7)

Next: Bootstrap cluster for the first time

Preparation for Lesson 8

Lesson 8 will go deeper into detailed Patroni configuration:

Analysis of each section in patroni.yml
Bootstrap configuration options
PostgreSQL parameters tuning
Authentication setup
Tags and constraints

Lesson 9: Bootstrap cluster

After Patroni is installed and configured, Lesson 9 will guide:

Starting Patroni for the first time
Automatic bootstrap process
Checking cluster status
Troubleshooting

Course

PostgreSQL High Availability A-Z

Share this article

You might also like