sum by (gen_ai_request_model) (
rate(gen_ai_client_token_usage_sum[1h]))Which model the token spend goes to.
A per-model rate, so a cost jump has a name attached before anyone opens a dashboard.
Token spend and model latency, per call.
GenAI calls become spans and metrics through the OpenTelemetry SDK, using the standard semantic conventions. Model, operation, token counts and finish reason arrive on every call, so a cost jump has a cause you can point at.
Included on the free tier: 20,000 active series, 50 GB of logs and 50 GB of traces a month. No card.
gpt-4o1.42M
in 1.06M · out 360k · 18.2k calls
claude-sonnet684k
in 512k · out 172k · 4.9k calls
text-embedding-391k
in 91k · out 0 · 1.0k calls
What you get
Included on every plan, including the free tier.
No separate AI product and no extra meter. GenAI spans and metrics are billed as the traces and series they are.
Input p50210 tokens
Output p5044 tokens
Output / input0.34
How it gets there
Three steps, then it is reporting.
GenAI spans leave through the OTLP exporter your SDK already has, into the same agent as the rest of your telemetry.
Install the OpenTelemetry contrib package for your provider.
Run the app under auto-instrumentation, pointed at your xScaler endpoint.
Model calls arrive with tokens and finish reason attached.
Coverage follows the OpenTelemetry contrib packages for each provider and language. For a client they don't cover, set the same GenAI attributes on a span you create yourself.
# 01 · install
pip install opentelemetry-distro \
opentelemetry-exporter-otlp \
opentelemetry-instrumentation-openai-v2# 02 · run under auto-instrumentation
export OTEL_SERVICE_NAME=support-assistantexport OTEL_EXPORTER_OTLP_ENDPOINT="https://<your-xscaler-endpoint>"opentelemetry-instrument python app.pychat gpt-4o · 1.68 s · CLIENT
gen_ai.operation.name "chat"
gen_ai.request.model "gpt-4o"
gen_ai.usage.input_tokens 3412
gen_ai.usage.output_tokens 486
gen_ai.response.finish_reasons ["stop"]
What you can ask
What that looks like as a query.
Plain PromQL against the GenAI metrics. Paste any of these into Insights, or use them as an alert rule.
rate(gen_ai_client_token_usage_sum{gen_ai_token_type="output"}[1h])
/ rate(gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[1h])Output against input.
Where prompt changes show. A system-prompt edit that doubles answer length moves this line first.
histogram_quantile(0.95, sum by (le, gen_ai_request_model)
(rate(gen_ai_client_operation_duration_bucket[5m])))The p95 a user actually waits.
Model latency on its own, separate from the retrieval and database work in the same request.
sum by (gen_ai_request_model) (
increase(gen_ai_client_token_usage_sum{gen_ai_token_type="input"}[30d]))
* 0.0000025Token counts, priced at your rate.
We store tokens, not dollars. Enter your contract rate and the multiplier in the query updates.
per million input tokens = a token
Questions
What AI teams ask before instrumenting.
Something not here? Talk to an engineer who works on the platform.
Book thirty minutesNo. GenAI spans leave through the OTLP exporter your SDK already has, into the same agent, and read back next to the rest of your telemetry. They are billed as the traces and series they are.
Start on the free tier.
20,000 active series, 50 GB of logs and 50 GB of traces a month, 90-day retention, and the whole of Insights.