Self-Hosting n8n with PostgreSQL & Docker for Enterprise AI Workflows
TL;DR: CrewAI is fastest to ship, AutoGen is most flexible, LangGraph is most reliable at scale. — the table below saves you hours, then we unpack each option.
Deploying automated AI agents and enterprise workflow pipelines on public cloud IPaaS platforms (such as Zapier, Make.com, or cloud-hosted n8n) often introduces severe operational boundaries: strict execution timeouts, data privacy / GDPR compliance violations, limited worker memory, and skyrocketing monthly operation fees. Self-hosting n8n inside an enterprise Virtual Private Cloud (VPC) provides complete infrastructure control, zero-trust security isolation, unlimited execution throughput, and native deep integration with local LLMs, vector databases, and enterprise relational databases.
However, running n8n in production requires moving far beyond the default single-process SQLite container deployment. High-volume AI workflows--such as multi-modal document parsing, high-concurrency webhook ingestion, and iterative RAG search--demand a production-grade **Distributed Queue Architecture** powered by **Docker Compose, PostgreSQL 16, Redis 7, Traefik v3 reverse proxy, and scalable worker pools**.
This technical deployment guide provides an end-to-end architecture blueprint, complete executable configuration manifests (`docker-compose.yml`, `.env`, `postgresql.conf`, `redis.conf`), automated database backup & pruning scripts, security hardening policies, zero-downtime deployment automation scripts, Prometheus observability setups, and performance scale-out playbooks.
Infrastructure Topology: Standalone SQLite vs. Distributed Queue Mode
Understanding n8n's internal execution architecture is essential before deploying to production. n8n supports two distinct operational topologies:
Standalone Single-Process Architecture (Non-Production)
In default standalone deployments (`EXECUTIONS_MODE=regular`), a single Node.js process handles all incoming webhooks, visual canvas UI rendering, task execution, and SQLite database reads/writes. When an AI workflow invokes a long-running LLM generation (e.g., taking 30 seconds to run a Llama-3 model or OpenAI call), the single Node.js event loop blocks. Under concurrent webhooks, request queues saturate, memory leaks accumulate, and the container experiences Out-Of-Memory (OOM) crash loops.
Enterprise Queue Mode Topology (Production Scale)
To support enterprise SLAs, n8n must be deployed in Queue Mode (`EXECUTIONS_MODE=queue`). This decouples infrastructure components into specialized microservices:
- n8n Main Container: Dedicated strictly to serving the web UI visual canvas, API endpoints, webhook triggers, and workflow scheduling. It doesn't execute heavy workflow nodes directly.
- Redis 7 Queue: Acts as an in-memory message broker. Incoming webhooks are immediately acknowledged (sub-10ms latency) and pushed as task jobs into Redis BullMQ queues.
- n8n Worker Containers: Multi-tenant worker processes pull jobs concurrently from Redis. Workers can be scaled horizontally ($1 \rightarrow N$ instances) across multiple cloud servers or container nodes based on CPU/RAM load.
- PostgreSQL 16 Database: Primary persistent relational datastore for workflows, credentials, execution histories, node states, and audit logs.
- Traefik v3 Reverse Proxy: Manages SSL/TLS automated Let's Encrypt certificate renewals, rate limiting, and secure edge routing.
Complete Production Docker Compose Infrastructure Manifest
The following production-grade `docker-compose.yml` manifest deploys a fully isolated, production-hardened n8n Queue Mode cluster with Traefik, PostgreSQL 16, Redis 7, and multiple scalable worker services equipped with explicit CPU and memory resource boundaries.
version: '3.8'
networks:
n8n-network:
driver: bridge
volumes:
traefik_letsencrypt:
postgres_storage:
redis_storage:
n8n_storage:
services:
# ---------------------------------------------------------------------------
# 1. Traefik v3: Edge Reverse Proxy & Automatic Let's Encrypt TLS
# ---------------------------------------------------------------------------
traefik:
image: traefik:v3.0
container_name: n8n_traefik
restart: always
command:
- "--api.insecure=false"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.web.http.redirections.entryPoint.to=websecure"
- "--entrypoints.web.http.redirections.entryPoint.scheme=https"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.tlschallenge=true"
- "--certificatesresolvers.myresolver.acme.email=${SSL_EMAIL}"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- traefik_letsencrypt:/letsencrypt
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '1.0'
memory: 512M
# ---------------------------------------------------------------------------
# 2. PostgreSQL 16: Relational Execution Datastore
# ---------------------------------------------------------------------------
postgres:
image: postgres:16-alpine
container_name: n8n_postgres
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- postgres_storage:/var/lib/postgresql/data
- ./postgres_config/postgresql.conf:/etc/postgresql/postgresql.conf
command: postgres -c config_file=/etc/postgresql/postgresql.conf
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
interval: 5s
timeout: 5s
retries: 5
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '2.0'
memory: 4096M
# ---------------------------------------------------------------------------
# 3. Redis 7: High-Speed Execution Queue Broker
# ---------------------------------------------------------------------------
redis:
image: redis:7-alpine
container_name: n8n_redis
restart: always
command: redis-server --requirepass ${REDIS_PASSWORD} --appendonly yes
volumes:
- redis_storage:/data
healthcheck:
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
interval: 5s
timeout: 5s
retries: 5
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '1.0'
memory: 1024M
# ---------------------------------------------------------------------------
# 4. n8n Main Service: UI Canvas & Orchestration Engine
# ---------------------------------------------------------------------------
n8n-main:
image: docker.n8n.io/n8nio/n8n:latest
container_name: n8n_main
restart: always
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- QUEUE_BULL_REDIS_PORT=6379
- QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
- N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
- N8N_HOST=${N8N_DOMAIN}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- WEBHOOK_URL=https://${N8N_DOMAIN}/
- N8N_DEFAULT_BINARY_DATA_MODE=filesystem
- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- N8N_METRICS=true
- N8N_METRICS_PREFIX=n8n_
volumes:
- n8n_storage:/home/node/.n8n
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n.rule=Host(`${N8N_DOMAIN}`)"
- "traefik.http.routers.n8n.entrypoints=websecure"
- "traefik.http.routers.n8n.tls.certresolver=myresolver"
- "traefik.http.services.n8n.loadbalancer.server.port=5678"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '1.5'
memory: 2048M
# ---------------------------------------------------------------------------
# 5. n8n Worker Pool Instance 1: Multi-Thread Worker Process
# ---------------------------------------------------------------------------
n8n-worker-1:
image: docker.n8n.io/n8nio/n8n:latest
container_name: n8n_worker_1
restart: always
command: worker
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- QUEUE_BULL_REDIS_PORT=6379
- QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
- N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
- N8N_DEFAULT_BINARY_DATA_MODE=filesystem
volumes:
- n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '2.0'
memory: 2048M
# ---------------------------------------------------------------------------
# 6. n8n Worker Pool Instance 2: Horizontal Scale Worker
# ---------------------------------------------------------------------------
n8n-worker-2:
image: docker.n8n.io/n8nio/n8n:latest
container_name: n8n_worker_2
restart: always
command: worker
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- QUEUE_BULL_REDIS_PORT=6379
- QUEUE_BULL_REDIS_PASSWORD=${REDIS_PASSWORD}
- N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
- N8N_DEFAULT_BINARY_DATA_MODE=filesystem
volumes:
- n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- n8n-network
deploy:
resources:
limits:
cpus: '2.0'
memory: 2048M
Production Environment Configuration (`.env`)
Create a secure `.env` file in the same directory as `docker-compose.yml`. Ensure strict permissions (`chmod 600 .env`) so unauthorized local users can't read master encryption keys.
# =============================================================================
# Production n8n Enterprise Environment Variables
# =============================================================================
# Domain & SSL Settings
N8N_DOMAIN=n8n.yourcompany.com
[email protected]
# PostgreSQL Credentials
POSTGRES_USER=n8n_db_admin
POSTGRES_PASSWORD=Secr3t_P0stgres_Pass_2026_Secure!
POSTGRES_DB=n8n_production
# Redis Password
REDIS_PASSWORD=Secr3t_R3dis_Pass_2026_Key!
# Master Encryption Key (32 Hex Characters for AES-256 Credential Vault)
# Generate with: openssl rand -hex 16
N8N_ENCRYPTION_KEY=9f8a7b6c5d4e3f2a1b0c9d8e7f6a5b4c
Database & Cache Tuning: High-Throughput Setup
By default, PostgreSQL out-of-the-box settings are configured for minimal system RAM (128MB). Under continuous execution tracking, n8n writes heavy JSON execution trees, causing VRAM thrashing and slow query locks. Place the following `postgresql.conf` performance tuning file inside `./postgres_config/postgresql.conf`.
Production `postgresql.conf` Tuning Matrix (Target Hardware: 4 vCPU, 16GB RAM)
# ------------------------------------------------------------------------------
# PostgreSQL 16 Optimized Configuration for n8n Queue Mode
# Target Spec: 16 GB System RAM, 4 vCPU SSD Server
# ------------------------------------------------------------------------------
# Memory Configuration
shared_buffers = 4GB # 25% of total system memory
work_mem = 64MB # Memory allocated for sort operations
maintenance_work_mem = 512MB # Memory for VACUUM and INDEX builds
effective_cache_size = 12GB # 75% of total RAM for query planner
# Write-Ahead Log (WAL) Optimization
wal_level = replica
max_wal_size = 8GB
min_wal_size = 1GB
checkpoint_completion_target = 0.9 # Spread WAL checkpoints evenly
# Worker & Connection Settings
max_connections = 200 # Accommodate workers and poolers
max_worker_processes = 4 # Match vCPU core count
max_parallel_workers_per_gather = 2
# Storage Write Optimization
synchronous_commit = off # Speed up JSON execution writes
random_page_cost = 1.1 # Optimized for NVMe SSD storage
Automated Execution Pruning SQL Script
If execution histories are not purged, the `execution_entity` table will grow to millions of rows, degrading n8n canvas UI response times. Create an automated SQL cron maintenance job using `pg_cron` or a daily Linux system cron script that retains executions for only 14 days.
-- SQL Pruning Script for n8n Execution Records
-- Deletes execution records older than 14 days to preserve NVMe disk space
BEGIN;
-- Delete binary data references for expired executions
DELETE FROM execution_data
WHERE "executionId" IN (
SELECT id FROM execution_entity
WHERE "stoppedAt" < NOW() - INTERVAL '14 days'
);
-- Delete execution entities
DELETE FROM execution_entity
WHERE "stoppedAt" < NOW() - INTERVAL '14 days';
COMMIT;
-- Reclaim unallocated disk space
VACUUM ANALYZE execution_entity;
VACUUM ANALYZE execution_data;
Automated S3 Database Backup Bash Script
Deploy this automated shell script (`/opt/scripts/n8n_backup.sh`) to perform daily zero-downtime hot backups using `pg_dump` and upload compressed archives to S3-compatible cloud storage.
#!/bin/bash
set -eo pipefail
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
BACKUP_DIR="/var/backups/n8n"
FILENAME="n8n_db_backup_${TIMESTAMP}.sql.gz"
S3_BUCKET="s3://company-n8n-backups-vault/postgres/"
mkdir -p ${BACKUP_DIR}
echo "[*] Starting PostgreSQL Backup for n8n..."
docker exec n8n_postgres pg_dump -U n8n_db_admin -d n8n_production | gzip > ${BACKUP_DIR}/${FILENAME}
echo "[*] Encrypting & Uploading to S3 Object Storage..."
aws s3 cp ${BACKUP_DIR}/${FILENAME} ${S3_BUCKET}${FILENAME}
echo "[*] Cleaning up local backups older than 7 days..."
find ${BACKUP_DIR} -type f -name "*.sql.gz" -mtime +7 -delete
echo "[✓] Backup Completed Successfully: ${FILENAME}"
Automated Zero-Downtime Deployment & Healthcheck Script (`deploy.sh`)
Deploy this initialization shell script (`/opt/scripts/deploy.sh`) to automate Docker stack deployments, verify environment variable integrity, pull latest container images, and execute database migration checks automatically.
#!/bin/bash
set -eo pipefail
echo "================================================================="
echo " Automated n8n Enterprise Cluster Deployment & Verification "
echo "================================================================="
if [ ! -f .env ]; then
echo "[!] ERROR: .env file missing in current working directory!"
exit 1
fi
source .env
if [ -z "$N8N_ENCRYPTION_KEY" ] || [ "${#N8N_ENCRYPTION_KEY}" -lt 16 ]; then
echo "[!] ERROR: N8N_ENCRYPTION_KEY must be set and at least 16 characters!"
exit 1
fi
echo "[*] Pulling updated Docker container images..."
docker-compose pull
echo "[*] Starting PostgreSQL & Redis Services..."
docker-compose up -d postgres redis
echo "[*] Waiting for PostgreSQL database initialization..."
until docker exec n8n_postgres pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB} > /dev/null 2>&1; do
sleep 2
echo " Waiting for PostgreSQL..."
done
echo "[✓] PostgreSQL is healthy and accepting connections."
echo "[*] Launching n8n Main Orchestrator & Worker Containers..."
docker-compose up -d --remove-orphans
echo "[*] Verifying Docker Container Status..."
docker-compose ps
echo "================================================================="
echo "[✓] n8n Enterprise Cluster Deployed Successfully at https://${N8N_DOMAIN}"
echo "================================================================="
Enterprise Monitoring & Observability Integration
For mission-critical production environments, monitoring worker queue depth, node execution latency, and PostgreSQL query locks prevents silent workflow outages. n8n exposes native Prometheus metrics when `N8N_METRICS=true` is set.
Prometheus Scraper Configuration (`prometheus.yml`)
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'n8n_production'
metrics_path: '/metrics'
scheme: 'https'
basic_auth:
username: 'metrics_user'
password: 'Secr3t_Metrics_Password'
static_configs:
- targets: ['n8n.yourcompany.com']
Key Prometheus Metrics Monitored in Grafana Dashboards:
- `n8n_instance_events_total`: Total counter of node executions categorized by status (`success`, `error`).
- `n8n_queue_jobs_waiting`: Number of jobs waiting in Redis BullMQ queues. Sudden spikes indicate worker pool under-provisioning.
- `n8n_workflow_execution_duration_seconds`: Histogram measuring end-to-end workflow completion latency.
Architectural Deployment Comparison
| Deployment Architecture | Max Concurrency | Fault Tolerance / HA | Ops Maintenance | Infrastructure Cost | Recommended Workload |
|---|---|---|---|---|---|
| Standalone Docker (SQLite) | 1 - 5 requests/sec | Zero (Single Point of Failure) | Minimal (Single container) | $5 - $10 / month | Local dev, hobby, light testing |
| Docker Compose Queue Mode (Postgres + Redis) | 200 - 500 requests/sec | High (Worker crash isolation) | Medium (Docker & SQL backups) | $25 - $80 / month | Enterprise Production SMB/Mid-market |
| Kubernetes Cluster (Helm + Managed Cloud Postgres) | 2,000+ requests/sec | Maximum (Horizontal Auto-Scaling) | High (DevOps K8s management) | $300+ / month | Large Global Enterprise / Multi-region |
Security Hardening & Zero-Trust Architecture Playbook
- Master Encryption Key Safeguards: The `N8N_ENCRYPTION_KEY` environment variable encrypts all API keys, OAuth tokens, and database credentials inside PostgreSQL using AES-256-GCM. Never alter or lose this key; losing it permanently corrupts all stored credentials.
- Code Node Sandboxing: Disable access to system environment variables and arbitrary Node.js modules inside visual Code nodes by restricting variable flags:
N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true NODE_FUNCTION_ALLOW_BUILTIN=* NODE_FUNCTION_ALLOW_EXTERNAL=lodash,axios,bignumber.js - Traefik IP Whitelisting & Basic Auth: Restrict webhook ingress or management endpoints using Traefik middleware headers to block automated bot scanning.
- PostgreSQL Network Isolation: Ensure PostgreSQL port `5432` and Redis port `6379` are NOT exposed to public host interfaces. They must remain accessible strictly within internal Docker bridge networks (`n8n-network`).
Edge Cases, Troubleshooting & Failure Modes
Redis Queue Staleness & Worker Deadlocks
- Symptom: Workflows remain indefinitely in the "Waiting" or "Executions Queued" state on the visual n8n canvas.
- Root Cause: n8n Worker containers lost TCP connection to Redis or hit OOM during a large payload processing, leaving orphan locks in BullMQ queues.
- Fix: Restart worker containers (`docker restart n8n_worker_1`) and issue a Redis memory flush command: `redis-cli -a $REDIS_PASSWORD EVAL "return redis.call('del', unpack(redis.call('keys', 'bull:n8n*')))" 0`.
PostgreSQL Connection Pool Exhaustion
- Symptom: Main container logs throw `FATAL: sorry, too many clients already`.
- Fix: Increase `max_connections` in `postgresql.conf` to 200, or deploy **PgBouncer** connection pooler in transaction mode between n8n workers and PostgreSQL.
Binary Data Memory Spikes during Large File Uploads
- Symptom: Worker containers crash when parsing multi-gigabyte video or PDF files.
- Fix: Ensure `N8N_DEFAULT_BINARY_DATA_MODE=filesystem` is configured. This instructs n8n to stream binary file payloads directly to disk storage rather than loading raw base64 buffer strings into Node.js V8 memory.
Last updated: September 1, 2026 -- reviewed for technical accuracy. Some benchmarks and API details evolve quickly; verify against the official docs linked below before production use.
Next step: Clone the repo, run the code above, and compare against your own data before trusting any benchmark.
Sources & Further Reading
Related on AI SaaS Edu
- Multi Agent State Persistence Architecture Redis PostgreSQL
- Deploying LiteLLM Proxy API Gateway with Auto Failover 2026 Guide
- Make.com vs n8n: Building Complex Conditional AI Agent Branching at Scale
Questions We Get Asked
How many n8n worker containers can I run on a single host?
Each n8n worker process consumes approximately 250MB to 512MB of base RAM, plus additional memory during heavy data transformations. As a rule of thumb, allocate 1 vCPU and 1GB of system RAM per n8n worker container. A host with 4 vCPUs and 16GB RAM can comfortably execute 8 worker instances alongside PostgreSQL and Redis.
How do I scale n8n across multiple physical servers?
To scale beyond a single VM host, deploy PostgreSQL and Redis on managed cloud infrastructure (e.g. AWS RDS PostgreSQL & AWS ElastiCache Redis). You can then launch n8n worker containers across multiple independent server nodes pointing to the central DB and Redis endpoints.
What happens if an n8n worker container crashes mid-execution?
Because n8n Queue Mode relies on Redis BullMQ, jobs are tracked with heartbeat acknowledgments. If a worker container crashes while processing an LLM call or API payload, the job times out in Redis and is automatically reassigned to another active worker process based on retry settings.
How do I configure custom npm packages inside self-hosted n8n?
Mount a custom node volume in `docker-compose.yml` (`- n8n_storage:/home/node/.n8n`), set the environment variable `NODE_FUNCTION_ALLOW_EXTERNAL=package_name`, and run `npm install package_name` inside `/home/node/.n8n` within the main and worker containers.
Can I use SQLite in production if I have low traffic?
SQLite is strictly a single-file datastore that doesn't support concurrent write locks. Even under moderate traffic, concurrent write attempts cause SQLite database locks (`SQLITE_BUSY: database is locked`), resulting in dropped webhooks. PostgreSQL 16 is mandatory for production stability.
Architectural Conclusion
Self-hosting n8n using Docker Compose, PostgreSQL 16, and Redis queue mode transforms n8n into an enterprise-grade automation engine. By decoupling the canvas UI orchestrator from distributed background worker pools, organizations gain sub-millisecond webhook ingestion, complete zero-trust data privacy, resilience against traffic surges, and total ownership over enterprise AI execution infrastructure.