> ## Documentation Index > Fetch the complete documentation index at: https://docs.platform.aiplanet.com/llms.txt > Use this file to discover all available pages before exploring further. # Models and providers > Map stable aliases such as fast, smart, vision, and embed to concrete provider deployments. Applications use stable aliases. Operators map those aliases to concrete provider deployments. There are two administration layers: * Deployment configuration owns providers, credential environment variables, and the baseline model catalog. * The private console and Admin API can add, update, delete, and alias concrete models that use an already configured provider. These changes are versioned in Postgres and survive restart. The repository audit and smoke scripts help operators select and validate routes, but they are not another configuration layer. `scripts/audit_model_catalog.py` does not add models or providers; UI mutations call the same Admin API used by automation. ## Discover available aliases ```bash theme={null} curl -sS "$GATEWAY_URL/v1/models" \ -H "Authorization: Bearer $GATEWAY_API_KEY" curl -sS "$GATEWAY_URL/v1/model/info" \ -H "Authorization: Bearer $GATEWAY_API_KEY" ``` `/v1/model/info` reports capabilities, safe routing shape, metadata, reviewed pricing, and client-safe audio formats, voice aliases, defaults, and input limits for models allowed to the key. It does not expose provider credentials or provider-native voice/format identifiers. ## Typical aliases | Alias | Intended use | | -------- | ------------------------------------------ | | `fast` | Low-latency, cost-conscious chat and tools | | `smart` | Higher-quality reasoning and generation | | `vision` | Image and multimodal requests | | `embed` | Text embeddings | These are deployment conventions, not globally guaranteed model mappings. Always discover the aliases enabled in your environment. ## Self-managed provider configuration Providers reference credentials through environment variables: ```yaml theme={null} providers: openai: type: openai auth: env: OPENAI_API_KEY ``` Concrete deployments define capabilities and reviewed prices. Keep them internal: ```yaml theme={null} models: - name: openai-fast provider: openai litellm_model: openai/ visibility: internal capabilities: tools: true vision: true reasoning: true pricing: input_cost_per_million: "0.00" output_cost_per_million: "0.00" source: provider-price-page reviewed_at: "YYYY-MM-DD" ``` Replace the example prices with current reviewed provider prices before issuing budgeted keys. Every weighted or fallback route reachable by a budgeted alias must have pricing. ### Speech models and provider mappings Transcription and English-translation models require the corresponding capability, public input formats, byte/duration limits, and reviewed `input_audio_second` pricing. Translation currently uses the native OpenAI-compatible adapter; other provider types are rejected during catalog validation. OpenAI transcription routes use the pinned LiteLLM adapter, while Azure Speech and ElevenLabs use reviewed native adapters behind the same OpenAI-shaped gateway endpoint. Speech synthesis models require `text_to_speech: true`, public voice and output-format aliases, a common default voice, and reviewed `input_character` pricing. OpenAI built-in names can use identity mapping. Provider-specific or custom voice IDs should always use an explicit internal mapping: ```yaml theme={null} providers: elevenlabs: type: elevenlabs auth: env: ELEVENLABS_API_KEY models: - name: elevenlabs-stt provider: elevenlabs litellm_model: elevenlabs/scribe_v2 visibility: internal capabilities: audio_transcription: true audio: input_formats: [mp3, wav] max_input_bytes: 26214400 max_input_seconds: "3600" pricing: meters: - unit: input_audio_second cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" - name: elevenlabs-tts-fast provider: elevenlabs litellm_model: elevenlabs/eleven_flash_v2_5 visibility: internal capabilities: text_to_speech: true audio: output_formats: [mp3, pcm] output_format_ids: mp3: mp3_44100_128 pcm: pcm_24000 voices: [default, warm] default_voice: default voice_ids: default: warm: pricing: meters: - unit: input_character cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" ``` The one `ELEVENLABS_API_KEY` secret can serve both models. The transcription adapter streams the bounded upload to ElevenLabs Scribe v2 without retaining raw audio. It accepts `language`, `temperature`, `json`, `verbose_json`, and word timestamps; `prompt` and segment timestamps are rejected before provider spend. Use limits and pricing reviewed for the selected account rather than copying the placeholders. `voices` and `output_formats` are the only names accepted from applications. `voice_ids` and `output_format_ids` are selected after routing and are never returned by the client API. Every target in a weighted/fallback TTS route must expose a common public voice and format. ElevenLabs and Azure Speech models fail catalog validation unless every public alias has an internal mapping. ### Realtime profiles Realtime aliases preserve one explicit provider-native event protocol. Configure the provider secret in deployment YAML, then add the concrete model and alias through YAML, the Admin API, or **Models → Add model / Add alias**. #### OpenAI Realtime ```yaml theme={null} providers: openai: type: openai auth: env: OPENAI_API_KEY models: - name: openai-realtime provider: openai litellm_model: openai/gpt-realtime-2.1 visibility: internal capabilities: realtime: true audio: realtime_protocol: openai-realtime-v1 pricing: meters: - unit: session_second cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" virtual_models: - name: realtime visibility: public weighted: - model: openai-realtime weight: 100 api_keys: - id: realtime-application token_env: REALTIME_APPLICATION_KEY allowed_models: [realtime] allowed_endpoints: - /v1/gateway/info - /v1/models - /v1/model/info - /v1/realtime # Add only when a trusted backend mints browser WebSocket sessions. - /v1/realtime/browser-sessions ``` Replace the model identifier and price with values validated for the provider account. Every weighted/fallback target must use the same `openai-realtime-v1` protocol; ElevenLabs and Deepgram native events cannot be placed behind this alias. The console exposes the capability, protocol, and typed price. It does not accept provider secrets. Production uses Redis-backed expiring per-key admission leases. Budgeted sessions reserve a bounded duration slice and renew ahead of consumption instead of holding the configured maximum; key revocation, admission loss, budget exhaustion, or persistence loss closes the active session fail-closed. #### ElevenLabs realtime STT The ElevenLabs profile relays Scribe v2 Realtime audio-to-transcript events. It is separate from bounded Scribe v2, realtime TTS `stream-input`, and ElevenAgents conversations: ```yaml theme={null} providers: elevenlabs: type: elevenlabs auth: env: ELEVENLABS_API_KEY models: - name: elevenlabs-stt-realtime provider: elevenlabs litellm_model: elevenlabs/scribe_v2_realtime visibility: internal capabilities: realtime: true audio: input_formats: [pcm_16000, ulaw_8000] realtime_protocol: elevenlabs-realtime-stt-v1 pricing: meters: - unit: session_second cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" virtual_models: - name: stt-realtime visibility: public weighted: - model: elevenlabs-stt-realtime weight: 100 ``` Allowed input formats are `pcm_8000`, `pcm_16000`, `pcm_22050`, `pcm_24000`, `pcm_44100`, `pcm_48000`, and `ulaw_8000`; configure only formats validated for the application. The client may select a configured format, language, timestamps, language detection, manual/VAD commit, and no-verbatim mode. Provider tokens, logging/retention controls, keyterms, and VAD thresholds remain deployment/provider-policy concerns rather than client-controlled query fields. The gateway does not retain frames, but provider-side handling follows the ElevenLabs account agreement and settings. #### ElevenLabs realtime TTS Realtime TTS streams partial text in and base64 audio events out. Keep its alias separate from bounded `/v1/audio/speech`, Scribe realtime STT, and ElevenAgents: ```yaml theme={null} models: - name: elevenlabs-tts-realtime provider: elevenlabs litellm_model: elevenlabs/eleven_flash_v2_5 visibility: internal capabilities: realtime: true audio: output_formats: [pcm, mp3] voices: [default, narrator] default_voice: default voice_ids: default: REPLACE_WITH_PROVIDER_VOICE_ID narrator: REPLACE_WITH_PROVIDER_VOICE_ID output_format_ids: pcm: pcm_16000 mp3: mp3_44100_128 realtime_protocol: elevenlabs-realtime-tts-v1 pricing: meters: - unit: input_character cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" virtual_models: - name: tts-realtime visibility: public weighted: - model: elevenlabs-tts-realtime weight: 100 ``` Every target must provide complete private mappings for the same public voice and output-format aliases. Realtime TTS reserves the configured maximum text characters at admission, rejects text beyond that cumulative bound, and settles to the accepted character count. Clients cannot override upstream model, provider voice ID, voice settings, provider logging, or credentials. #### ElevenLabs Agents Conversational agents use their own signed-URL protocol. The private deployment model is the provider agent ID; the public alias exposes neither that ID nor the short-lived signed token: ```yaml theme={null} models: - name: elevenlabs-voice-agent provider: elevenlabs litellm_model: elevenlabs/REPLACE_WITH_PRIVATE_AGENT_ID visibility: internal capabilities: realtime: true audio: realtime_protocol: elevenlabs-agent-v1 pricing: meters: - unit: session_second cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: provider-price-page reviewed_at: "YYYY-MM-DD" virtual_models: - name: voice-agent visibility: public weighted: - model: elevenlabs-voice-agent weight: 100 ``` The gateway calls ElevenLabs' signed-URL endpoint with the provider credential, validates that the result remains on the configured provider origin and is bound to the configured agent, then opens the WebSocket without exposing either credential. Client query parameters cannot select an agent, branch, environment, or token. Agent prompt, LLM, voice, dynamic-variable defaults, retention, and provider logging stay deployment/account owned. Use separate aliases and keys for agents with different tenant or data-handling boundaries. ### Azure Speech STT and TTS Azure Speech is a native region-scoped adapter. Add the provider to deployment YAML first; the Admin API and console never accept the subscription key or region. One environment-backed Speech resource key can serve both STT and TTS models in that region: ```yaml theme={null} providers: azure-speech: type: azure-speech region: eastus2 auth: env: AZURE_SPEECH_KEY models: - name: azure-speech-stt provider: azure-speech litellm_model: azure-speech/stt visibility: internal capabilities: audio_transcription: true audio: input_formats: [wav] max_input_bytes: 10485760 max_input_seconds: "60" pricing: meters: - unit: input_audio_second cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: azure-speech-pricing reviewed_at: "YYYY-MM-DD" - name: azure-speech-tts provider: azure-speech litellm_model: azure-speech/tts visibility: internal capabilities: text_to_speech: true audio: output_formats: [mp3, wav] output_format_ids: mp3: audio-24khz-48kbitrate-mono-mp3 wav: riff-16khz-16bit-mono-pcm voices: [default] default_voice: default voice_ids: default: en-US-AvaNeural pricing: meters: - unit: input_character cost_per_unit: "REPLACE_WITH_REVIEWED_COST" source: azure-speech-pricing reviewed_at: "YYYY-MM-DD" ``` Replace the region, voice, formats, limits, and prices with values reviewed for your resource. The current Azure short-STT adapter deliberately accepts only valid 16 kHz, 16-bit, mono PCM WAV input and requires a model limit of at most 60 seconds. It supports `json` and `verbose_json`, defaults the language to `en-US`, and rejects prompt, temperature, and timestamp parameters before provider spend. Azure TTS supports configured public voice/format mappings and speed from `0.5` through `2.0`. The Admin API and **Models → Add model** console form accept these fields. In the console, public formats and voices belong under **Audio contract**; provider mappings remain visible only on the private operator surface. Typed pricing fields cover input/output audio seconds, input characters, and realtime session seconds. Provider credentials remain YAML/secret-manager owned in all cases. Self-managed deployments may define a fixed `headers` map on a provider for upstream tenant or routing headers. Those values are applied by both gateway engines and are never accepted from application requests or returned by the public model APIs. Virtual aliases support weighted targets and configured fallbacks. Run real chat, streaming, tools, vision, Responses, embeddings, transcription, translation, or speech checks for each declared capability before making it available to applications. ## Add a model and alias in the console 1. Open `/admin/ui`, authenticate, and select **Models**. 2. Add a concrete model using a configured provider and its LiteLLM model identifier. 3. Declare only capabilities that have been tested. For audio, enter the public formats/voices, private provider mappings, required limits, and the matching typed price meter. Mark its validation state. 4. Add any unique alias name and choose one or more concrete targets with positive weights. 5. Create an API key and select that public alias. 6. Call `/v1/models` and `/v1/model/info`, verify the alias's client-safe audio contract, then send an inference request using the alias. Names may contain letters, digits, `.`, `_`, `:`, and `-`, up to 128 characters. Concrete models are normally internal; aliases are normally public. Disabled or internal entries cannot be granted to application keys. ## Add a model and alias with the API First read the active revision: ```bash theme={null} curl -sS "$GATEWAY_ADMIN_URL/admin/model-catalog" \ -H "Authorization: Bearer $GATEWAY_ADMIN_KEY" ``` Create a priced concrete model using that revision: ```bash theme={null} curl -sS -X POST "$GATEWAY_ADMIN_URL/admin/models" \ -H "Authorization: Bearer $GATEWAY_ADMIN_KEY" \ -H 'Content-Type: application/json' \ -d '{ "revision": 0, "name": "provider-model-a", "provider": "openai", "litellm_model": "openai/", "visibility": "internal", "capabilities": {"tools": true, "vision": false, "reasoning": true, "embeddings": false}, "pricing": { "input_cost_per_million": "1.00", "output_cost_per_million": "4.00", "source": "provider-price-page", "reviewed_at": "YYYY-MM-DD" }, "metadata": {"validation": "configured"} }' ``` Use the returned revision to create any application-facing alias: ```bash theme={null} curl -sS -X POST "$GATEWAY_ADMIN_URL/admin/aliases" \ -H "Authorization: Bearer $GATEWAY_ADMIN_KEY" \ -H 'Content-Type: application/json' \ -d '{ "revision": 1, "name": "invoice-agent-model", "visibility": "public", "weighted": [{"model": "provider-model-a", "weight": 100}], "metadata": {"intent": "invoice extraction"} }' ``` Create the application key with `allowed_models: ["invoice-agent-model"]`. A key scope must match an active public model or alias; a misspelling is rejected instead of creating a key that can never work. ## Authority and replica behavior After the first API mutation, the Postgres catalog is authoritative over the YAML `models` and `virtual_models` sections. `/admin/reload` and process restart reapply that database catalog. `/admin/model-catalog/reset` explicitly returns to the current deployment YAML baseline. If the stored graph cannot produce a ready provider engine, `GET /admin/model-catalog` reports `source: database_pending` and preserves a compatible graph for inspection. The process retains its last safe runtime, `/ready` returns 200 with `status: degraded`, `serving_ready: true`, and `model_catalog_ready: false`, and Prometheus reports the model-catalog component as down. Repair or reset the pending revision while the prior runtime continues serving. The response includes `schema_version` and `supported_schema_version`. When a stored catalog was written by a newer incompatible gateway, older replicas report `model_catalog_schema_unsupported`, keep serving their safe runtime, and reject catalog edits or reload so they cannot overwrite fields they do not understand. Upgrade the replica or use revision-based reset when discarding the stored override is intentional. A catalog mutation or reset advances a Postgres generation. Every replica polls, readiness-checks, and atomically activates that generation, then publishes its acknowledgement. `GET /admin/model-catalog` includes generation, local generation, poll/stale timing, and every known replica's status. Configure a unique stable `LLM_GATEWAY_INSTANCE_ID` on each process and monitor component health. A lagging or failed process reports degradation but continues serving its last known-good runtime.