Shared responsibility
The deployment owner is responsible for gateway patching, infrastructure, provider credentials, network separation, monitoring, backup/restore, and incident response. Application owners are responsible for key distribution, prompt/data policy, and choosing scopes, limits, and budgets appropriate to each workload.Secrets
- Store provider keys, database/Redis URLs, bootstrap keys, the bootstrap administrator password, application tokens, and webhooks in a secret manager.
- Never put secret values in gateway YAML, source control, container images, screenshots, tickets, or logs.
- Application tokens are stored as hashes; the raw generated token is returned once.
- Organization BYOK credential documents are encrypted as a whole with a dedicated vault key. During rotation, explicitly configured previous keys provide a bounded read overlap and startup re-encrypts their documents with the current key; stored plaintext secret fields fail closed.
- Optional browser Realtime credentials are short-lived, exact-scope, single-use, hash-stored, and sent through the WebSocket subprotocol rather than a URL. The trusted backend must return them with no-store handling and the ingress must not log
Sec-WebSocket-Protocol. - Optional provider-direct WebRTC returns an opaque
gwrtc_bootstrap credential, never the upstream ephemeral credential. Its Redis record is single-use and short-lived. Minting is backend-only;/v1/realtime/callspermits only explicitly configured browser origins. - Rotate atomically with a bounded overlap, switch the application secret, validate, and then invalidate the prior generation or revoke the key.
Network separation
Expose only inference/discovery paths publicly. Keep/ready, /metrics, /admin/*, API documentation, and the console private. The supplied public Nginx proxy enforces this allowlist for the Compose deployment.
Data handling
By default, the gateway database stores request metadata, attribution, status, token counts, cost estimate, latency, and model/provider routing without prompt or response bodies. Payload persistence defaults todisabled. If explicitly enabled, credential-redacted bounded input/output is available only through tenant-scoped request details and is cleared on its own retention schedule; ordinary PII remains visible. A leased worker also enforces request-metadata and administrator-audit retention and removes expired sessions, OIDC transactions, and prior rotation hashes. See Data retention and privacy.
An explicit local troubleshooting switch can emit bounded prompt, response, and stream-chunk events to stdout. It is disabled by default. If enabled, terminal/container/collector logs contain sensitive application content and must follow the deployment’s access, deletion, and retention policy.
Vision URLs and inline images pass to the configured provider. The gateway does not fetch, scan, or retain managed attachments. Your provider and application data-handling agreements still apply.
Audio transcription uploads are temporary provider-operation inputs. They are bounded, spooled, closed after the request, and excluded from both text payload capture modes. Generated speech is streamed through without entering payload observers or durable storage. Bounded TTS input follows the deployment’s text payload setting; realtime frames, including TTS text chunks, are always excluded from payload capture. Durable request records may contain observed duration or character count, status, and cost, but never raw audio or realtime frame content. Provider-side handling still follows the selected provider and deployment data policy.
Browser Realtime offers two deliberately different trust modes. The gateway-terminated WebSocket keeps parent-key revocation, admission, budgets, retention, metrics, and shutdown enforceable for the whole session. The optional provider-direct WebRTC bootstrap relays authentication and SDP only; media and data channels then go directly to the provider. It is disabled by default, restricted to exact configured origins and one concrete OpenAI-compatible route, and rejects budgeted keys or payload-retained deployments. Do not represent that lower-governance path as gateway-observed or continuously controlled.
Current administration boundary
- Named administrators use eight-hour opaque
HttpOnly,SameSite=Strictsessions. - Each login receives an independent CSRF token bound to that session; missing, static, and stale-session tokens are rejected for mutations.
- Passwords are memory-hard hashed. Raw passwords, session tokens, and CSRF tokens are never persisted.
- Roles separate system administrators/operators from organization administrators/operators/viewers. Organization roles require exactly one tenant scope, and invalid or unknown persisted roles are rejected before authorization.
- Password and OIDC initiation attempts are distributed-throttled through Redis.
- OIDC Authorization Code + PKCE accepts only explicitly mapped immutable provider subjects; IdP group or role claims never grant gateway permissions.
- Optional AI Planet Platform handoff uses short-lived signed, single-use tokens and creates a distinct organization-scoped principal per Platform membership; tokens are submitted by form POST rather than URLs or browser storage.
- Admin bearer keys remain available for scripts and automation, not browser login.