vllm-project/semantic-router

feat: add backend analytics read model for routing quality dashboards

Open

#2,368 opened on Jul 5, 2026

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/dashboardarea/observabilityenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Add a backend analytics read model/API that aggregates routing quality, distribution, latency, signal confidence, and replay-derived summaries for dashboard consumption.

Parent roadmap: #2287 Parent issue: #2345 Related: #2157, #2200, #2326, #2330, #2333 Related code: dashboard/backend, dashboard/backend/evaluation, dashboard/backend/configprojection, src/semantic-router/pkg/routerreplay, src/semantic-router/pkg/observability

Motivation

Dashboard analytics should not parse raw config or raw replay blobs in the frontend. The backend needs a maintained read model that normalizes route distribution, model selection, latency, signal confidence, and low-confidence/unrouted summaries.

Scope

Implement backend APIs for:

  • route/model/recipe distribution;
  • latency and overhead summaries;
  • signal confidence and projection summaries;
  • links to replay detail records;
  • redaction/privacy controls;
  • query windows and aggregation bounds.

Non-goals

  • Do not store raw prompts in aggregate analytics by default.
  • Do not invent dashboard-only routing semantics.
  • Do not replace #2333 eval reports.

Acceptance criteria

  • Frontend dashboard views can consume normalized analytics APIs.
  • Aggregates are backed by replay/metrics/config snapshots with redaction.
  • API tests cover privacy and summary behavior.

Contributor guide