vllm-project/semantic-router

feat: add dashboard analytics for routing quality, drift, and unrouted traffic

Open

#2,345 opened on Jul 5, 2026

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Go (699 forks)github user discovery
area/dashboardarea/observabilityarea/user-experienceenhancementhelp wantedpriority/P2roadmap

Repository metrics

Stars
 (4,293 stars)
PR merge metrics
 (PR metrics pending)

Description

Summary

Add dashboard analytics for routing quality and operations, including route distribution, latency metrics, embedding or signal drift, and unrouted query clustering.

Parent roadmap: #2287 Related: #1509, #1519, #1610, #2200, #2326, #2330, #2333

Child issues

  • #2368: Add backend analytics read model for routing quality dashboards.
  • #2369: Add frontend routing analytics views for dashboard.
  • #2370: Add drift and unrouted-query clustering with privacy controls.

Motivation

Adopters need to understand how traffic is being routed, where routing is failing, and which new routes or signal improvements are needed. Today, debug and replay data exist across router replay, response headers, metrics, evaluation storage, config projection, and dashboard APIs, but the dashboard does not provide a unified analytics view for route quality and drift.

Scope

Dashboard analytics should cover:

  • backend read model/API for normalized analytics (#2368);
  • frontend views for route/model/recipe distribution (#2369);
  • latency and overhead by route/model/recipe;
  • signal confidence and drift trends;
  • unrouted or low-confidence query clustering (#2370);
  • decision and projection explainability summaries;
  • links from aggregate views to replay/debug records;
  • privacy and retention rules for aggregates and samples.

Non-goals

  • Do not invent a separate dashboard-only routing contract.
  • Do not store raw sensitive prompts without respecting existing privacy and retention rules.
  • Do not replace the decision-level eval harness.

Acceptance criteria

  • Dashboard surfaces can display routing distribution and quality signals from maintained backend APIs.
  • Analytics are backed by normalized config/dump/replay/metrics data, not ad hoc frontend parsing.
  • Unrouted or low-confidence traffic can be summarized for recipe and signal improvement work.
  • Docs explain data retention and privacy considerations.

Contributor guide