runtime: introduce generation-owned construction, reload, and graceful shutdown
#2,470 opened on Jul 12, 2026
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Summary
Introduce an immutable runtime generation that owns every closeable child, rolls back partial construction, leases active requests and streams, and drains safely on reload or process shutdown. Owner surfaces: src/semantic-router/pkg/extproc/router.go, src/semantic-router/pkg/extproc/router_build.go, src/semantic-router/pkg/extproc/server.go, src/semantic-router/pkg/routerruntime/, src/semantic-router/pkg/services/, and src/semantic-router/cmd/runtime_bootstrap.go.
Current behavior
Router construction has no complete rollback stack, reload replaces and immediately closes the prior router without a request lease, classification refresh can publish mixed state, and process shutdown drains only a subset of resources. Expanding close behavior without ownership and leases would create use-after-close risk.
Expected behavior
One versioned generation owns every resource, construction commits only after full success, each request observes one generation, and retirement stops admission, drains active work to a deadline, then closes children exactly once in reverse order.
Acceptance
- Own caches, classification snapshots, response and Replay stores, memory, rate-limit clients, workflow state, selectors, native schedulers, vector runtime, and servers in one generation.
- Use a reverse cleanup stack for constructor failure and join cleanup errors.
- Acquire and release generation leases for requests and streams; publish immutable classification state atomically.
- On reload, stop old admission, drain leases, retire native and MCP resources, and close idempotently.
- On process shutdown, stop servers and queued admission, drain work, close children, and synchronize hook registration.
Validation
- Fault-inject each constructor step and prove exact-once reverse cleanup.
- Run reload and classification refresh against long-lived requests under
go test -race. - Assert stable goroutine, descriptor, connection, and native-handle counts across repeated reloads.
- Add startup and loaded shutdown tests with bounded drain and no loss of acknowledged state.
- Run the router-core feature gate and affected AMD runtime profiles.
Related
- Parent audit: #2375
- Native runtime tracker: #2396
- Durable architecture: TD045 and TD020