1. Home
  2. ...
  3. Metrics
  4. AI Gateway metrics

AI Gateway metrics

What each chart on the AI Gateway dashboard shows and how to filter the data.

The AI Gateway view of the Metrics screen shows the consumption, latency, cost, and governance of the AI calls that go through AI Gateway.

IMPORTANT

This view only has metrics for those who have AI Gateway contracted.

Filter

  • Period: when it opens, the view shows the last hour.
  • Agent: in All agents, choose an agent and click Filter to see only its metrics. Clear goes back to all agents.
  • MCP tool: in the Governance & MCP section, choose a tool in All MCP tools and click Filter.

Traffic & Tokens

Traffic & Tokens section with the call and token consumption charts

Chart or table
What it shows
LLM calls over timeTotal calls to models per interval
Token consumption trendTokens per second, split into
input, cache read, and output
HTTP status of AI routesProportion of successful and error responses
Cache hit-rate by provider / modelPercentage of prompt tokens served
by the cache, by provider and model
Calls by provider / modelVolume of calls by provider and model
Avg tokens per callAverage tokens per call, split
into input, output, and cache
Tokens by provider / model / typeTotal tokens by provider, model, and type

Latency

Latency section with the latency charts by model

Chart
What it shows
Latency percentilesp50, p90, p95, and p99 latency over the period
Average latency by modelAverage end-to-end
response time, by model
Time to first token (TTFT)Time to the first token of the response, by model
Tokens per second (TPOT)Output tokens per second, by model

Cost

Cost section with the cost per hour, by model, and by agent

Chart or table
What it shows
Cost per hour by providerSpend per hour, in dollars, by provider
Cost by provider / modelSpend in the period, by provider and model
Cost by agent (chargeback)Spend attributed to each agent
Cost per request & per 1K tokensCost per call and per thousand tokens, by model

Governance & MCP

Governance & MCP section with the guardrail and MCP tool usage charts

Chart or table
What it shows
Guardrail blocks over timeRequests and responses
rejected by guardrails
Evaluations by action and phaseGuardrail evaluations by action, in
the request and response phases
Guardrail interventions vs requestsGuardrail interventions relative
to authenticated requests
MCP calls over timeCalls to MCP tools per interval
MCP usage by server / toolTotal calls by MCP
server and tool

To understand an empty chart or a chart at zero, see No data and zero value.

How happy are you with this page?

We use cookies to enhance your experience on our site. By continuing to browse, you agree to our use of cookies.Learn more