Advanced Setup

The advanced steps below are intended for people with sysadmin experience. If you are not comfortable with these steps, please refer to the basic setup guide.

Introduction

For the advanced setup, first follow the basic setup guide to get the server up and running. Once you have the server running, you can follow the steps below to configure the server for your specific needs.

Configuration

Setting config via environment variables

The server uses environment variables to store configs. You can set these environment variables in a .env file in the root of the project. The .env file should look like this:

# .env
KEY1=value1
KEY2=value2

The server will automatically load the .env file when it starts. You can also set the environment variables directly in your shell. Refer to your operating system's documentation on how to set environment variables in the current session.

The valid options are listed in .env.default in the root of the builder and server directories. You can copy the .env.default file to .env and modify the values as needed.

# Copy the .env.default file to .env
cp .env.default .env

Secrets directory

The secret directory is located at ./secrets. You can store any secrets you need in this directory. The server will automatically load the secrets when it starts.

An example for a secret called my_secret would look like this:

# ./secrets/my_secret
my_secret_value

This is useful when running on docker so you can copy the secrets into the container without exposing them in the Dockerfile.

Database selection

PostgreSQL

We use PostgreSQL (with the pgvector extension) as the database. You will swap the commands you use to generate and run prisma to the following

poetry run prisma generate --schema postgres/schema.prisma

This will generate the Prisma client for PostgreSQL. You will also need to run the PostgreSQL database in a separate container. You can use the docker-compose.yml file in the rnd directory to run the PostgreSQL database.

cd autogpt_platform/
docker compose up -d --build

You can then run the migrations from the backend directory.

cd ../backend
prisma migrate dev --schema postgres/schema.prisma

Redis

The platform uses Redis for caching, distributed locks, rate limits, and the streams that carry agent output to the browser.

The Docker Compose stack runs its own Redis Cluster: three redis:7 shards on ports 17000 to 17002, joined by a one-shot redis-init service. make start-core starts it along with the other dependencies.

Using an external Redis cluster

The backend connects with a cluster client only, so the deployment must have cluster mode enabled. A standalone server, with cluster mode disabled, does not work.

The cluster also needs:

• Redis 7.0 or later, or a compatible engine such as Valkey. The backend uses sharded pub/sub (SPUBLISH, SSUBSCRIBE), which Redis 7.0 introduced, along with Streams and Lua scripts. It uses no Redis modules.
• Shard addresses the backend can reach. With REDIS_USE_ANNOUNCED_ADDRESS=true, the backend connects to each shard at the address the cluster announces, so those addresses must resolve from where the backend runs. Without it, the backend connects to every shard at REDIS_HOST and keeps only the announced port, which works only when every shard is reachable on that one host.
• A private network between the backend and every shard. The backend connects without TLS and does not send a username. It sends REDIS_PASSWORD when one is set, and a cluster without a password works too. Nothing on the connection is encrypted, so don't route it over a network you don't trust.

Point REDIS_HOST and REDIS_PORT at any node of the cluster, and set REDIS_PASSWORD if the cluster has one.

In the Compose stack, REDIS_HOST, REDIS_PORT and REDIS_USE_ANNOUNCED_ADDRESS are set in the x-backend-env block of autogpt_platform/docker-compose.platform.yml, which takes precedence over backend/.env, so change them there. REDIS_PASSWORD is read from backend/.env. The bundled shards still start, because the backend services wait for redis-0 to be healthy.

AutoGPT Agent Server Advanced set up

This guide walks you through a dockerized set up, with an external DB (postgres)

Setup

We use the Poetry to manage the dependencies. To set up the project, follow these steps inside this directory:

1. Install Poetry

   pip install poetry

2. Configure Poetry to use .venv in your project directory

   poetry config virtualenvs.in-project true

3. Enter the poetry shell

   poetry shell

4. Install dependencies

   poetry install

5. Copy .env.default to .env

   cp .env.default .env

6. Generate the Prisma client

   poetry run prisma generate

   In case Prisma generates the client for the global Python installation instead of the virtual environment, the current mitigation is to just uninstall the global Prisma package:

   pip uninstall prisma

   Then run the generation again. The path should look something like this: <some path>/pypoetry/virtualenvs/backend-TQIRSwR6-py3.12/bin/prisma

7. Run the postgres database from the /rnd folder

   cd autogpt_platform/
   docker compose up -d

8. Run the migrations (from the backend folder)

   cd ../backend
   prisma migrate deploy

Running The Server

Starting the server directly

Run the following command:

poetry run app