AI Operations

Every time Orion answers a question, an AI Analyst runs, or any other AI feature fires, Celiq records one row in the ai_calls telemetry log — its surface, provider, model, token counts, cost, latency, and whether it succeeded. The AI Operations tab reads that log and rolls it up into headline stats, a daily cost trend, per-surface and per-model breakdowns, and a filterable list of recent calls.

Because this view exposes spend and operational health across the whole workspace, it is restricted to Luminary admins.

Overview

AI Operations is a single dashboard scoped to a time window you choose: Last 7 days, Last 30 days, or Last 90 days (30 days by default). Changing the window reloads every panel.

The page is laid out top to bottom:

• Headline stats — six cards summarizing the whole window.
• Daily cost trend — one bar per day, colored by whether that day had errors.
• By surface — each AI feature (surface) with its calls, tokens, latency, cost, and error count.
• By model — each provider + model pairing with its calls and cost.
• Recent calls — a paginated, filterable list of individual AI calls.

If no AI calls were recorded in the selected window, the page shows a short note instead: usage, cost, and latency appear once Orion answers a question or an AI feature runs.

When to use it

• Cost monitoring — Check the Est. cost card and the daily cost trend to see what AI is costing this week, this month, or this quarter.
• Finding the expensive surface — Use the By surface breakdown to see which feature drives the most spend or the most tokens.
• Confirming which model is live — Use the By model breakdown to confirm the provider and model your workspace is actually calling.
• Reliability checks — Watch the Success rate and Failures cards. A daily bar turning amber flags a day with errors; the By surface panel shows which feature failed.
• Debugging a specific failure — Filter the Recent calls list to Error status to read the error message attached to each failed call.

Concepts

Getting started

• You must be signed in as a Luminary (admin). The tab does not appear for Weaver, Tracer, or Lens roles, and the underlying data endpoints return an error for non-Luminary users.
• At least one AI call must have been recorded in your chosen window for the panels to show data.

1. Open Context Studio.
2. In the left navigation, find the Governance group.
3. Click AI Operations.

Step-by-step

Examples

The panels read directly from the ai_calls telemetry log. Conceptually, the headline summary is the equivalent of:

SELECT
  COUNT(*)                                    AS calls,
  SUM(input_tokens)                           AS input_tokens,
  SUM(output_tokens)                          AS output_tokens,
  SUM(total_tokens)                           AS total_tokens,
  SUM(cost_usd)                               AS cost_usd,
  ROUND(AVG(latency_ms))                       AS avg_latency_ms,
  COUNT(*) FILTER (WHERE status = 'ok')        AS ok,
  COUNT(*) FILTER (WHERE status = 'error')     AS errors
FROM ai_calls
WHERE workspace_id = :workspace
  AND created_at >= :window_start
  AND created_at <  :window_end;

Success rate is then computed as ok / calls for the window. You never run this query yourself — it backs the headline cards automatically.

Best practices

• Pick the window that matches your question. Use 7 days to spot a sudden change, 30 days for routine cost review, and 90 days for trend and budgeting.
• Read cost and reliability together. A cheap day with errors is not a healthy day — check the Failures card and amber bars alongside cost.
• Start broad, then drill in. Use the headline stats and the by-surface breakdown to find what changed, then filter Recent calls to confirm the detail.
• Use the by-model panel after model changes. After updating your AI configuration, confirm here that calls are landing on the model you expect.

Tips

Common mistakes

Troubleshooting

Related pages

• AI Hub
• Usage
• Skill Registry
• Billing