> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.aiplanet.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy with Docker Compose

> Run the supported Docker Compose topology on a private VM in your own cloud account.

Run the supported Docker Compose topology on a private Linux VM in your cloud account. Applications reach only a TLS reverse proxy or tunnel; administrators and Prometheus use the private loopback gateway port.

## Architecture

* Gateway, Postgres, Redis, and public Nginx proxy on one host.
* Public TLS endpoint forwards to `127.0.0.1:4082`.
* Private operator access reaches `127.0.0.1:4000` through SSH, VPN, or a private network.
* Postgres and Redis have no host ports in the baseline Compose file.

## Install

```bash theme={null}
git clone https://github.com/aiplanethub/llm-gateway.git
cd llm-gateway
cp deploy/compose/.env.example deploy/compose/.env
```

Edit `deploy/gateway.yaml`:

1. Set `public_base_url` to the external HTTPS origin.
2. Give the environment a unique `shared_state.key_prefix`.
3. Keep only the providers and model routes you intend to operate.
4. Review model capabilities, unsupported parameters, and pricing snapshots.
5. Keep concrete provider deployments internal and expose stable aliases.

Populate `deploy/compose/.env` from your secret manager. Do not commit it.

| Variable                               | Purpose                                                                                                                              |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `GATEWAY_CONFIG_FILE`                  | Host path to the environment configuration; defaults to `deploy/gateway.yaml` through `../gateway.yaml` relative to the Compose file |
| `POSTGRES_PASSWORD`                    | Password used to initialize the bundled Postgres service                                                                             |
| `LLM_GATEWAY_DATABASE_URL`             | Async URL such as `postgresql+asyncpg://gateway:<url-encoded-password>@postgres:5432/gateway`                                        |
| `LLM_GATEWAY_REDIS_URL`                | Keep `redis://redis:6379/0` for the bundled Redis service                                                                            |
| `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` | Credentials for the providers retained in `deploy/gateway.yaml`                                                                      |
| `GATEWAY_BOOTSTRAP_CLIENT_KEY`         | Initial application token; replace it after provisioning                                                                             |
| `GATEWAY_BOOTSTRAP_ADMIN_KEY`          | Initial private operator token; replace or tightly restrict it                                                                       |
| `GATEWAY_BOOTSTRAP_ADMIN_PASSWORD`     | Initial `gateway-admin` console password; at least 12 characters                                                                     |

The optional monitoring overlay also uses
`LLM_GATEWAY_NOTIFICATION_PROVIDER` (`slack` or `google_chat`) and
`LLM_GATEWAY_NOTIFICATION_WEBHOOK_FILE`, which must point to a mode-`0600` host
file containing one approved HTTPS incoming-webhook URL. The file is mounted
read-only rather than copied into the container environment. Local Compose does
not remap bind-mounted file ownership: keep the parent directory root-only and
own the mode-`0600` file as numeric UID/GID `10001:10001` so the non-root
gateway and relay can read it.

Generate independent random bootstrap values, for example with `openssl rand -hex 32`. If you remove a provider from `deploy/gateway.yaml`, also remove its required environment entry from the `gateway` service in `deploy/compose/gateway.yaml`.

Open `http://127.0.0.1:4000/admin/ui` over the operator tunnel and sign in as `gateway-admin` using `GATEWAY_BOOTSTRAP_ADMIN_PASSWORD`. The bootstrap admin API key is for scripts such as the stage gate; do not paste it into the console.

The supplied loopback-only deployment sets `admin_session_cookie_secure: false` so cookies work through that HTTP SSH tunnel. Set it to `true` when the private operator origin itself uses HTTPS; never expose the HTTP operator origin beyond loopback or a trusted private tunnel.

```bash theme={null}
docker compose --env-file deploy/compose/.env \
  -f deploy/compose/gateway.yaml up --build --wait
```

To include private Prometheus and Alertmanager services, select the receiver and
enable the matching `alerts.slack_webhook_url_file` or
`alerts.google_chat_webhook_url_file` field in the gateway YAML, then add
`-f deploy/compose/observability.yaml` to the command. Their host ports default
to loopback-only 4092 and 4093.

## Expose safely

Configure your TLS proxy, load balancer, or tunnel to forward only to `127.0.0.1:4082`. Never point it to port 4000. Verify:

```bash theme={null}
curl -fsS https://gateway.example.com/health
test "$(curl -sS -o /dev/null -w '%{http_code}' https://gateway.example.com/admin/config)" = 404
```

For an internal pilot where every application is already authorized on the same Tailscale tailnet, `sudo python3 scripts/configure_tailscale_https.py --apply` creates and verifies a tailnet-only HTTPS/WSS origin on the node's stable MagicDNS name. Use the reported HTTPS origin as `public_base_url`. This is not public ingress and must not use Tailscale Funnel; external customer traffic requires a managed TLS endpoint and a separate ingress review.

From the operator network:

```bash theme={null}
curl -fsS http://127.0.0.1:4000/ready
curl -fsS http://127.0.0.1:4000/metrics | head
```

For a remote VM, create a local operator tunnel instead of opening port 4000:

```bash theme={null}
ssh -L 4000:127.0.0.1:4000 operator@gateway-host
```

## Validate before use

Run the disposable-key stage gate with the public and private origins:

```bash theme={null}
uv sync --extra dev --frozen
export LLM_GATEWAY_ADMIN_KEY="$GATEWAY_BOOTSTRAP_ADMIN_KEY"
uv run --extra dev python scripts/stage_gateway_smoke.py \
  --gateway-url https://gateway.example.com \
  --admin-url http://127.0.0.1:4000 \
  --model fast
```

Require `status: passed`, `key_revoked: true`, and `operator_surface: verified_private`. Then create one key per application/environment and run a real task before migrating traffic.

No certified VM size is published yet. Measure your models, payloads, streaming duration, and concurrency with representative load before production use.