feat: add ROCm ATOM inference-aware backend adapter
#2,352 opened on Jul 5, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Summary
Add a ROCm ATOM adapter for inference-aware backend routing so AMD self-hosted deployments can expose ATOM runtime, cache, and profiling signals through the same backend policy used by other engines.
Parent roadmap: #2287
Parent issue: #2332
Depends on: engine-neutral backend telemetry contract
Related external project: https://github.com/ROCm/ATOM
Related code: src/semantic-router/pkg/config, src/semantic-router/pkg/selection, src/semantic-router/pkg/latency, deploy/amd, deploy/operator, src/fleet-sim
Motivation
ATOM is a ROCm-focused lightweight vLLM-like implementation with OpenAI-compatible serving, prefix caching, P/D disaggregation, profiling, and benchmark surfaces. Those capabilities are directly relevant to self-hosted AMD routing, but they should enter vLLM Semantic Router through a normalized engine adapter rather than through ATOM-specific routing branches.
Scope
Define and implement ATOM support for:
- ATOM backend/replica identity under a logical model;
- OpenAI-compatible serving endpoint integration;
- queue, latency, GPU, health, prefix-cache, and disaggregation-related telemetry where available;
- ROCm/AMD deployment examples under
deploy/amdor related Kubernetes profiles; - diagnostics that distinguish ATOM telemetry freshness and fallback from vLLM/SGLang.
Non-goals
- Do not require ATOM for default CPU CI.
- Do not duplicate ATOM's benchmark dashboard inside the router.
- Do not make AMD-specific telemetry fields part of the generic backend policy unless they are normalized.
Acceptance criteria
- ATOM backends can be modeled as backend-aware candidates without changing logical recipe/decision behavior.
- ATOM telemetry maps to the normalized backend telemetry contract.
- Missing telemetry falls back to existing routing behavior.
- AMD/ATOM examples document opt-in setup and validation expectations.