ops: revisit Docker and Kubernetes backend endpoint support completeness

Summary

Revisit the completeness of vllm-sr Docker and Kubernetes deployment support for backend endpoint configuration, Kubernetes endpoint auto-discovery, and dashboard editing/visibility.

Parent roadmap: #2287 Related: #2331, #2332, #2349, #2350, #2351, #2352, #2353, #2355, #2358, #2373, #2374 Related code: src/vllm-sr/cli, src/vllm-sr/cli/config_generator.py, src/vllm-sr/cli/config_translator.py, src/semantic-router/pkg/config/canonical_providers.go, src/semantic-router/pkg/k8s, deploy/operator/api/v1alpha1, deploy/operator/controllers/backend_discovery.go, deploy/operator/controllers/canonical_config_builder.go, deploy/helm, deploy/local, deploy/recipes, dashboard/backend/handlers/config.go, dashboard/backend/handlers/deploy.go, dashboard/backend/handlers/status*.go, dashboard/frontend/src/components/EndpointsEditor.tsx, dashboard/frontend/src/pages/ConfigPage*.tsx

Motivation

H2 adds multiple recipes, backend-aware routing, engine-specific adapters, edge deployments, and richer provider capability contracts. Those features depend on a consistent way to express and manage backend targets across local Docker, Kubernetes/operator, Helm, recipes, and dashboard flows.

The current code already has several related surfaces:

• canonical provider models use providers.models[].backend_refs[] with fields such as endpoint, base_url, type, provider, protocol, chat_path, and auth/header fields;
• Docker/CLI Envoy generation reads backend.endpoint or backend.base_url and parses host, port, protocol, and path;
• operator vllmEndpoints[].backend.type currently supports Kubernetes-native discovery types such as kserve, llamastack, and service;
• dashboard endpoint editing currently exposes a narrower endpoint shape such as name, address/endpoint, protocol, and weight.

We need to make the supported matrix explicit and then close the gaps so users can move the same routing config between Docker, Kubernetes, operator-managed deployments, and dashboard editing without losing backend intent.

Scope

Audit and improve support for:

• Docker/local vllm-sr serve

  • canonical backend_refs using direct endpoint targets such as host:port;
  • URL-style targets using base_url or full URL strings, including protocol and path handling;
  • external provider backends that need provider type, base URL, auth header, API key env, API version, custom chat path, or extra headers;
  • generated Envoy cluster behavior for DNS, static IPs, HTTPS, path prefixes, and multiple endpoints.

• Kubernetes/operator/Helm

  • CRD coverage for backend target kinds beyond the current service discovery enum where needed;
  • auto-discovery for Kubernetes-native endpoint types, including direct Service references, label-based services, KServe, Llama Stack, and future endpoint-style targets;
  • whether EndpointSlice/Endpoints or service annotations should be used for richer discovery when a Service alone is insufficient;
  • canonical translation from discovered Kubernetes objects into providers.models[].backend_refs[] without losing protocol, path, auth, weight, model alias, or provider metadata;
  • Helm values translation parity with the canonical config and operator contract.

• Dashboard

  • endpoint editor support for the full backend target contract instead of only the narrow endpoint/protocol/weight subset;
  • validation and preview behavior for both direct URL-style backends and Kubernetes-discovered endpoint types;
  • status/readiness display for configured and discovered backend targets;
  • safe editing without clobbering fields that the dashboard does not yet render.

• Documentation and examples

  • reference recipes that demonstrate direct endpoint, full URL/base URL, external provider, Kubernetes Service discovery, and operator-managed discovery;
  • a compatibility matrix showing which backend target kinds work in Docker, Helm, operator, dashboard, and examples.

Non-goals

• Do not implement inference-aware load balancing itself here; that remains covered by #2332 and its child issues.
• Do not redesign the full config schema here unless a narrow compatibility gap requires a schema addition.
• Do not turn dashboard editing into an unsafe free-form YAML rewrite that silently drops unknown fields.
• Do not require every Kubernetes platform integration to land in one PR; split implementation gaps into child issues if needed.

Acceptance criteria

• A backend target compatibility matrix exists for Docker/local, Helm, operator/Kubernetes, dashboard, and reference recipes.
• The matrix explicitly covers at least direct endpoint, full URL or base_url, external provider target, Kubernetes Service, label-discovered Service, KServe, Llama Stack, and any supported EndpointSlice/Endpoints discovery path.
• Gaps are split into focused child issues or PRs with affected surfaces and validation commands.
• Docker/CLI behavior for endpoint versus URL/base URL backend refs is documented and covered by targeted tests.
• Kubernetes/operator discovery either supports the agreed endpoint target kinds or documents unsupported kinds with clear validation errors.
• Dashboard config editing preserves backend target fields that it does not render, and renders/validates the agreed supported fields.
• Reference recipes and docs show how to configure the same logical model backend in local Docker and Kubernetes/operator paths where applicable.
• The work cross-links implementation issues with #2332/#2349 backend-aware routing contracts when discovery data becomes runtime telemetry input.

Validation

Use the repo harness to scope follow-up PRs:

make agent-report ENV=cpu CHANGED_FILES="src/vllm-sr/cli src/semantic-router/pkg/config src/semantic-router/pkg/k8s deploy/operator deploy/helm deploy/local deploy/recipes dashboard/backend dashboard/frontend/src"
make agent-lint CHANGED_FILES="src/vllm-sr/cli src/semantic-router/pkg/config src/semantic-router/pkg/k8s deploy/operator deploy/helm deploy/local deploy/recipes dashboard/backend dashboard/frontend/src"
make agent-ci-gate CHANGED_FILES="src/vllm-sr/cli src/semantic-router/pkg/config src/semantic-router/pkg/k8s deploy/operator deploy/helm deploy/local deploy/recipes dashboard/backend dashboard/frontend/src"

Add targeted tests for CLI Envoy generation, operator backend discovery, CRD/webhook validation, Helm translation, dashboard backend config preservation, and frontend endpoint editing as implementation gaps are fixed.