Policies

The Policies module defines what AI Gateway does with each request that reaches a route before forwarding it to the LLM Provider, and what it does with the returned response. Each policy is linked to a single route and combines three optional blocks: Prompt Guard (filtering by moderation or regex), Prompt Enrichment (messages injected before the prompt), and Rate Limit (quotas by requests or tokens).

IMPORTANT

You must configure at least one section to save the policy. The form lets you combine Prompt Guard (Request), Prompt Guard (Response), Prompt Enrichment, and Rate Limit in any arrangement, but at least one must be filled in.

Quick links

Learn how to:

• Filter and list policies
• View a policy
• Create a policy
• Edit a policy
• Delete a policy
• Review the route field
• Configure Prompt Guard (Request)
• Configure Prompt Guard (Response)
• Configure Prompt Enrichment
• Configure Rate Limit
• Review the required permissions

Listing

The Policies page lists registered policies with pagination. You can filter by:

• Route (optional): route associated with the policy.

Use Search to apply the filter and Clear to reset it.

Columns shown in the table:

• Route: associated route.
• Configured sections: indicates which sections are configured (Prompt Guard / Prompt Enrichment / Rate Limit).
• Created At: creation date and time.
• Actions: View and Delete icons for each row.

View Policy

In the Actions column of the row you want, click the View icon. The Policy details screen opens in read-only mode, showing:

• Route, Configured sections, and Created At at the top of the card, before the sections.
• Below that, only the sections actually configured on that policy: Prompt Guard (Request), Prompt Guard (Response), Prompt Enrichment, and Rate Limit. A section that is not configured does not appear on the screen. The labels are those of the form, with these differences: the Prompt Enrichment rows read "Message N · role" and "Message N · content"; On detection shows "Monitor" where the form shows "Monitor only"; the regex rows appear only when a regex rule exists, and the patterns are shown one per line; with Lakera in Monitor only, the Policy details screen shows If the guardrail does not respond with the value "Let the request through", while the form hides that field; the rejection response is shown only when the block can reject, so it is hidden in Monitor mode without a Reject rule, even though it stays saved.

No field is editable on this screen. To change the policy, click the Edit button in the card header (see Edit Policy).

Create Policy

Edit Policy

Open the policy you want (see View Policy) and, on the Policy details screen, click the Edit button in the card header. The Update Policy form opens with the current data. Change the required fields and click Save.

Delete Policy

In the Actions column of the row you want, click the Delete icon. Confirm the operation in the "Do you want to delete policy [policyId]?" dialog.

IMPORTANT

Deletion is permanent and cannot be undone. Deleting a policy makes the associated route accept requests without guardrails, prompt enrichment, or rate limiting.

Required field

Prompt Guard (Request)

This optional block validates the request before it reaches the LLM provider. It has three parts, in the order the screen shows them: Guardrail (the moderation provider), Regex rules, and Rejection response.

PROVIDER AND REGEX CAN BE COMBINED

The regex rules run on every request, with or without a provider. When a provider and a regex are both set, the request must pass both checks, and the same rejection response applies to whichever one rejects it.

Provider

The Provider field opens the block and has three options, in this order: None, OpenAI Moderation, and Lakera. A new policy opens on None, and a saved policy opens on the provider it uses. A policy uses one provider at a time.

With None, the policy does no moderation: the screen shows no credential field, Test connection, On detection, timeout, or failure behavior, and only the regex rules apply. Choosing a provider declares moderation, and its fields become required.

The key selector lists only the API Keys of the chosen provider: OpenAI keys with OpenAI Moderation and Lakera keys with Lakera. The list is fetched again every time the form opens, so a key registered moments ago shows up without reloading the page.

If the key stored in the policy is no longer in the provider's list, for example because it was deleted, the field is cleared and the screen shows "The saved API key is no longer available for this provider. Choose another one.". On save, the screen asks for another key. If the tenant has no key for the provider, the empty state described below applies.

If the tenant has no key for the chosen provider yet, the selector gives way to an empty state: "This tenant has no API key with provider Lakera." (or "OpenAI"), a shortcut "Register a Lakera API key" (or "Register an OpenAI API key") to API Keys registration, and the note "The policy can only be saved after a key exists.". On save in that state, the screen shows "Register a Lakera API key before saving this policy." (or "Register an OpenAI API key before saving this policy."). The shortcut only works for users with the Create and Update API Keys permission.

Fields: Provider with Lakera

Fields: Provider with OpenAI Moderation

Action on detection

The On detection field applies to both providers, has two options, and opens on Reject:

• Reject: a request the provider flags is rejected, and the client receives the configured rejection response. With Lakera, the screen explains: Lakera reports what it found and you choose what the gateway does. With OpenAI Moderation, OpenAI returns a verdict and the gateway rejects the request.
• Monitor only: detection blocks nothing, and the verdict goes to the metrics in Observability. With Lakera, not even a provider failure blocks. With OpenAI Moderation, the request still fails when OpenAI does not answer.

With Lakera, the gateway makes the decision. The mode of your Lakera project (Enforce or Detect) does not change the outcome: in both, whatever Lakera detects counts as a detection, and what happens to the request is set by On detection on this policy.

Moderation has no option to mask content.

When the provider does not respond

The If the guardrail does not respond field sets what happens when the provider does not respond: because of a timeout, an outage, or an error. With Lakera and the Reject action, you choose between Block the request (the default) and Let the request through. With Lakera and Monitor only, the field is hidden. With OpenAI Moderation, the screen shows the read-only value Block the request, with the explanation that the gateway blocks when it cannot evaluate.

With Block the request, the route answers with an error while the provider does not respond: "The route answers with an error while the provider does not respond. Safety over availability.". The gateway itself answers with the error (503). The message and status of the rejection response apply to a detection, not to a provider failure.

With Let the request through, the screen shows an informational note: "While the provider is down, prompts reach the model without being screened. The route stays up; the protection does not."

Two cases with Lakera follow this same field: when the request is still too large for the provider even after leaving messages out, and when the request cannot be read. With Block the request, the route answers with the gateway error (503), as in a provider failure. With Let the request through, the request goes on to the model without being screened.

Long conversations are handled separately. With Lakera, when a conversation exceeds the size limit, the oldest messages are left out and the rest is analyzed normally. The system instructions and the last user message are always kept. This does not trigger the If the guardrail does not respond field, and the part that was left out is not checked.

IMPORTANT

Letting the request through when Lakera does not respond means the content is not checked during the outage. Blocking protects, but makes the route answer with an error while the provider is down.

Timeout

With Lakera, the Timeout (ms) field sets how long the gateway waits for the provider before giving up. The help text on the screen is "How long to wait for the provider before giving up. Maximum 9000 ms.". The field accepts whole numbers from 1 to 9000, and a value outside that range shows "Enter a whole number between 1 and 9000.". If the field is left empty, a default time applies.

With OpenAI Moderation, the screen shows the read-only value 10000: the gateway always waits 10000 ms, and this time is not configurable.

Test the connection

The Test connection button appears only with Lakera, right after Project ID and before On detection. It uses the key and the Project ID in the form and confirms that the gateway can reach Lakera with those credentials. The test runs only when you click, and it saves nothing. While it runs, the screen shows "Testing the connection...", and on success, "Connection successful. The provider answered in X ms.".

If the test fails, the screen shows the reason: credential rejected by the provider, provider unavailable, or timeout.

On save, the gateway runs the same check in two cases. The first is when the policy starts using Lakera, either by being created with it or by switching the provider to Lakera. The second is when a policy that already uses Lakera has its key or Project ID changed. If the check fails, the policy is not saved. Switching from Lakera to another provider does not trigger it. Changing anything else (rate limit, regex, prompt enrichment, On detection, the failure behavior, the timeout, or the rejection response) does not either. The check shows that the configuration worked at the moment of saving. If the provider goes down later, the behavior in when the provider does not respond applies.

Data sent to the provider

THE PROMPT CONTENT LEAVES THE GATEWAY

With Lakera, the prompt content of each request is sent to Lakera for analysis. Lakera is a provider you contract on your own: the contract, the account use, and the handling of data on their side are yours. The same applies to OpenAI Moderation and your contract with OpenAI. Check that this meets your organization's data policies before enabling it.

Scope

Moderation with Lakera analyzes the request only, before it reaches the LLM provider. The LLM response does not go through Lakera, and moderation does not mask content.

Regex rules

Commas in patterns. A comma inside (), {} or [] stays in the pattern, so (a,b), \d{2,5} and [a,b] are one pattern each. A comma outside them separates patterns when you type it at the end of the text with nothing selected, so typing foo, adds foo. Typed in the middle of foobar, it stays in the text, and pressing Enter adds the single pattern foo,bar. Inside brackets, ( and { are plain characters: [(] and [{] match a literal parenthesis or brace, and a comma after [{] separates patterns. To match a literal comma, write it as [,], which works in the form and in any regular expression engine. A backslash escapes the next character. Pasted text is not split: paste it and press Enter to add it as one pattern. On save, text still in the field is also added as a single pattern, without leading or trailing spaces and with any commas it has; blank text is ignored.

Rejection response

Response returned to the requester whenever the request is rejected, by the provider or by a regex rule.

Both fields are optional. The rejection response is saved when the block has a provider, in any action (Reject or Monitor only), or When a pattern matches is Reject with at least one pattern. In those cases the screen fills in whatever is left blank: a blank status becomes 400 and a blank message becomes the default text (with both blank, the screen fills in both). In any other case no rejection response is saved, even if you typed values in these fields.

A policy saved earlier with a status below 400 stays readable. Where the rejection response is saved (the cases above, a policy with a provider in Monitor only included), the screen only saves the policy again after you change the status to a value between 400 and 599. In any other case the stored response is dropped on save.

The rejection response applies to a detection. When the provider does not respond, the failure behavior applies.

Prompt Guard (Response)

This optional block validates the response returned by the provider before returning it to the requester. There is no moderation here, only regex and the rejection response, with the same rules as the request.

STREAMING RESPONSES

For now, only responses returned without streaming are masked by the expressions configured in this section.

Regex rules (response)

Rejection response (response)

Returned to the requester whenever the response is rejected.

The same rules as the request apply, with one difference: the response block saves the rejection response only when Response regex action is Reject with at least one pattern. In that case the screen fills in whatever is left blank. Otherwise no rejection response is saved, even if you typed values in these fields.

A policy saved earlier with a status below 400 can only be saved again from the screen after its status is changed to a value between 400 and 599, as long as the block saves the rejection response. Otherwise the stored response is dropped on save.

Prompt Enrichment

This block is optional as a section, but has required internal rules when enabled. When you add at least one message, you must provide at least one complete entry (role + content). Messages are prepended to the original prompt in the order they appear in the list.

Use Add message to add a new Message entry and the delete icon to remove entries.

Fields by message

AVAILABLE ROLES

system and developer are the only selectable roles. They are the ones that make sense for instructions prepended to the prompt. If a policy shows a different role when opened for editing, the form flags it with the message This role is no longer supported. Select "system" or "developer". and requires the change before saving.

Rate Limit

This optional block limits consumption by time window. It counts either tokens or requests, never both. The rate limit is saved only when Rate limit value is filled in; with the field empty, the policy has no rate limit.

MUTUAL EXCLUSION

A policy limits by tokens or by requests, never both at once. Choose one in Count by.

The screen shows Count by as radio buttons, Token (preselected) and Request, which determine what is counted.

Fields

Permissions

Access to each action in the Policies journey depends on its own permission. Without the corresponding permission, the screen element does not appear: the interface never shows an unavailable action as disabled.

Permissions are granted in Access Control, on the Roles screen, under the AI Gateway product. The table below lists each one exactly as Access Control displays it.

The last three belong to other journeys and are required by specific Policies actions, shown in the next table.

View: the (provider) suffix of the Moderation API Key and of the Lakera API Key additionally requires View API Key. Without it, the field shows only the bare id.

Test connection: the button requires Create and Update Policies, the same permission as creating and editing. Without it, the button does not appear.

Create and Edit: List Routes is what feeds the form's Route field with the available routes, and List API Keys is what feeds the moderation key selector.

EDITING REQUIRES VIEWING

The Edit button only exists inside the Policy details screen. Because of that, a user holding Create and Update Policies but not View Policy has no path to edit an existing policy. This is expected platform behavior, not a bug.