Self-Host Instructions

Self-host Engine on your own infrastructure for free and manage your self-hosted Engine from the thirdweb dashboard.

Requirements

  • Docker
  • A thirdweb secret key from the API Keys page
  • PostgresDB (version 14+)
  • Redis (version 7.2.4+)

Running Engine locally

Start Postgres:

docker run -p 5432:5432 -e POSTGRES_PASSWORD=postgres -d postgres

Start Redis:

docker run -p 6379:6379 -d redis:7.2.4

Start Engine server:

docker run \
-e ENCRYPTION_PASSWORD="<encryption_password>" \
-e THIRDWEB_API_SECRET_KEY="<thirdweb_secret_key>" \
-e ADMIN_WALLET_ADDRESS="<admin_wallet_address>" \
-e POSTGRES_CONNECTION_URL="postgresql://postgres:[email protected]:5432/postgres?sslmode=disable" \
-e ENABLE_HTTPS=true \
-e REDIS_URL="redis://host.docker.internal:6379/0" \
-p 3005:3005 \
--pull=always \
--cpus="0.5" \
thirdweb/engine:latest

Environment variables

Variable (* = Required)Description
ENCRYPTION_PASSWORD*Provide a string to encrypt sensitive data stored in DB. Do not change this value or encrypted data will be inaccessible.
THIRDWEB_API_SECRET_KEY*A thirdweb secret key created on the API Keys page.
ADMIN_WALLET_ADDRESS*The wallet address that will configure Engine from the thirdweb dashboard. You will be able to add other admins later.
POSTGRES_CONNECTION_URL*Postgres connection string: postgresql://[user[:password]@][host][:port][/dbname][?param1=value1&...]
REDIS_URL*Redis connection string: redis://[user:password@]host:port[/db-number]
ENABLE_HTTPSSelf-sign a certificate to serve API requests on HTTPS. Set to true if running Engine locally only. (Default: false)
LOG_LEVELDetermines the logging severity level. Adjust for finer control over logged information. (Default: debug)
PRUNE_TRANSACTIONSWhen false, Engine prevents the pruning/deletion of processed transaction data. (Default: true)
ENABLE_KEYPAIR_AUTHEnables Keypair Authentication.
TRUST_PROXYWhen true, trust the X-Forwarded-For header to allow Engine to use the correct client IP address for the IP allowlist.

* Required

Your server is running when this log line appears:

Listening on https://localhost:3005. Manage your Engine from https://thirdweb.com/dashboard/engine.

Manage Engine from the dashboard

To manage your Engine instance from the dashboard:

  • Navigate to https://localhost:3005/json

    • The "Your connection is not private" page will appear.
    • Select Show advanced and select Proceed to localhost (unsafe) to render the JSON file.
    • This one-time step allows your browser to connect to your local Engine instance.
  • Navigate to the Engine dashboard page.

  • Sign in with the <admin_wallet_address> wallet.

  • Select Import

  • Add your publicly accessible Engine URL.

    • If Engine is running locally, provide the URL https://localhost:3005.

Running Engine in production

See the Production Checklist for best practices using Engine in a production environment.

  • Pin Engine Docker to a version to control when changes are introduced.
    • latest may include major version changes which can introduce breaking changes.
  • Host Engine DB on any Postgres-compatible database.
    • Remember to update POSTGRES_CONNECTION_URL.
    • Examples: AWS RDS, Google Cloud SQL, Heroku, Supabase
    • Minimum specs: 2 vCPU, 2 GB memory (AWS equivalent: t4g.small)
    • Minimum version: 14
    • Add the ?connection_limit=10 param to your POSTGRES_CONNECTION_URL to increase the connection limit.
  • Host Engine server on any cloud provider.
    • Minimum specs: 1 vCPU, 2 GB memory (AWS equivalent: t2.small)
    • Auto-scale the instance count to increase inbound throughput and queuing capacity.
  • Remove the ENABLE_HTTPS env var.

FAQ

Why is my Docker image unable to reach my Postgres database?

Here are common troubleshooting tips:

  • Ensure the Postgres DB is running in Docker. POSTGRES_CONNECTION_URL should be set to localhost (if in the same container) or host.docker.internal (if in a different container).
  • Ensure the Postgres DB connection URL and credentials are correct.
  • Ensure the database name exists on the Postgres DB.

See Production Checklist for best practices using Engine in a production environment.

Self-hosting recommendations

  • Do not set the environment variable ENABLE_HTTPS=true.
  • Host Engine Docker on a cloud provider.
    • Minimum specs: 1 vCPU, 2 GB memory (AWS equivalent: t2.small)
    • Auto-scale the instance count to increase inbound throughput and queuing capacity.
  • Host Postgres DB on a cloud provider.
    • Examples: AWS RDS, Google Cloud SQL, Heroku, Supabase
    • Minimum specs: 2 vCPU, 2 GB memory (AWS equivalent: t4g.small)
    • Set the connection_limit parameter within your POSTGRES_CONNECTION_URL environment variable to 10.

How do I filter logs in Engine?

Configure log verbosity via the LOG_LEVEL environment variable. The severity levels ordered from highest to lowest are:

  • fatal : Terminates the program due to critical errors.
  • error : Highlights serious issues needing immediate action.
  • warn : Suggests caution due to potential issues.
  • info : Shares routine operational insights.
  • debug : Provides detailed debugging information.
  • trace : Offers in-depth tracing details.

Engine by default captures logs at debug severity and higher. Setting LOG_LEVEL to error limits logging to only error and fatal severities.

LOG_LEVEL="error"

How to prevent SIGSEGV errors with the Engine docker image?

Ensure the Engine docker image's stability by allocating a minimum of 0.5 vCPU & 1 GB memory to the container, mitigating SIGSEGV (Segmentation Fault) errors due to inadequate resources or memory access issues.

Engine recommends a minimum DB connections limit of 8 PER HOST. Please set the connection_limit parameter within your POSTGRES_CONNECTION_URL environment variable, to allow a connection limit between (8, # conns supported by your DB / # hosts). In practice, 10-20 is a suitable connection limit.

DB Error:

Timed out fetching a new connection from the connection pool. More info: http://pris.ly/d/connection-pool (Current connection pool timeout: 10, connection limit: 3)

Example configuration:

POSTGRES_CONNECTION_URL=postgres://postgres:postgres@localhost:5432/postgres?connection_limit=10

What is x-forwarded-for and how does it affect Engine?

If you have engine running on a server behind a reverse proxy, you can set the TRUST_PROXY environment variable to true to trust the X-Forwarded-For header. Reverse proxies like Nginx or Apache will add this header to the request with the original client IP address, and setting this variable will allow Engine to use the correct IP address for the IP Allowlist. For more details on IP Allowlisting, refer to the Security Features page.