LLM Providers

The LLM Providers module defines the set of models and providers that AI Gateway can invoke when a route is triggered. Each provider is a reusable configuration that groups one or more provider + model + API Key combinations with different priorities. This helps you define failover strategies or balance requests across models.

NOTE

Before creating an LLM Provider, register the API Keys for the providers you want to use. The form only lets you save when at least one key exists for each selected combination — the only exception is Bedrock, which also accepts Assume Role authentication.

Quick links

Learn how to:

• Filter and list LLM Providers
• Create an LLM Provider
• Edit an LLM Provider
• Delete an LLM Provider
• Review the form fields
• Understand conditional fields by provider
• Configure Bedrock with Assume Role

Listing

The LLM Providers page lists registered configurations with pagination. You can filter by:

• Provider (optional): associated LLM provider. Values: All Providers, OpenAI, Gemini, Anthropic, Azure OpenAI, Azure AI Foundry, Bedrock, Custom.
• Model (optional): text search by model name.

Use Search to apply the filter and Clear to reset it.

Columns shown in the table:

• Providers and Models: summary of registered Provider / Model / Priority combinations, separated by |.
• Created At: creation date and time.
• Actions: Edit and Delete icons for each row.

Create LLM Provider

Edit LLM Provider

In the Actions column of the row you want, click the Edit icon. The Update LLM Provider form opens with all configurations prefilled. Change the required fields, add or remove provider rows, and click Save.

Delete LLM Provider

In the Actions column of the row you want, click the Delete icon. Confirm the operation in the "Are you sure you want to delete this LLM provider?" dialog.

IMPORTANT

Deletion is permanent and cannot be undone. Routes that reference the deleted LLM Provider start failing. Update or remove dependent routes before deleting it.

Form fields

The form is made up of a list of provider configurations. At least one is required. Each Provider block repeats the fields in the table below.

Action buttons outside the table:

• Add provider: adds a new Provider block to the list.
• Delete icon for each block: removes the corresponding block (disabled when there is only one configuration).
• Save: saves the creation or edit. Cancel: discards changes.

Conditional fields by provider

The visibility and required state of some fields depend on the Provider selected in each block:

• OpenAI, Gemini, and Anthropic: do not require extra fields beyond Priority, Provider, Model, and API Key.
• Azure OpenAI: shows the required Resource name field.
• Azure AI Foundry: shows the Resource name and Project name fields, both required.
• Bedrock: shows the Region field (required) and the Guardrail identifier + Guardrail version pair (optional as a pair; when you fill in one, the other becomes required).
• Custom: shows the Host and Port fields, both required. Use this provider for any model that exposes an API compatible with the OpenAI completions standard (OpenAI-like) — the requirement applies both to your own or self-hosted models and to providers not yet natively supported by the platform.

TIP

The Guardrail pairing rule prevents ambiguous configurations. To disable the guardrail, clear both fields. They become optional again, and the set is not sent when you save.

Bedrock with Assume Role

For the Bedrock provider, the API Key field is optional. Leave it blank when you want the gateway to authenticate with AWS through the Assume Role mechanism, where credentials are resolved at the infrastructure level instead of being registered as a static key. This is the recommended setup for cross-account AWS integrations.

If you prefer to authenticate with a static key, select a Bedrock API Key as usual — both forms are accepted, and the choice applies per configuration block.

REQUEST THE INFRASTRUCTURE SETUP

Assume Role depends on an infrastructure setup that must be prepared in advance for your environment. Open a ticket with the Sensedia support team requesting Assume Role to be enabled for the Bedrock integration. Without this preparation, a Bedrock provider registered without an API Key will not be able to authenticate with AWS.