[RFC]: TTS Development Roadmap - Q2 2026

Motivation

Q1 established TTS as a first-class capability in vLLM-Omni: Qwen3-TTS reached production quality (RTF 0.34, TTFP 131ms), five TTS models are supported, and core infrastructure (async_chunk, streaming, voice management) is in place.

Q2 shifts from "make TTS work" to "make TTS easy, reliable, and composable." TTS models are small and fast. The challenge is no longer performance but how TTS integrates with the broader vLLM ecosystem.

Theme 1: Streaming as a Core Feature (P0)

Streaming is not optional for TTS. Users expect real-time audio output as text is being generated. This applies at every level:

Streaming text input: Accept text incrementally (from ASR, LLM output, or user typing) and begin synthesis before the full input is available. The WebSocket /v1/audio/speech/stream endpoint provides this today for Qwen3-TTS. This should be generalized to all TTS models and become a standard capability.

Streaming audio output: Chunk-based audio generation with low first-chunk latency. Already supported via async_chunk and HTTP chunked transfer. Needs to be reliable and consistent across all models.

End-to-end streaming: When combined with Theme 2 (composable layer), this means text LLM token generation streams directly into TTS synthesis with minimal buffering. The full pipeline from prompt to audio should stream without waiting for complete text generation.

Every new TTS model integration should support streaming from day one. Non-streaming should be the fallback, not the default.

Theme 2: TTS as a Composable Layer (P0)

Today, speech output requires a model with a built-in talker stage. But the most common production ask is: "I already have a text LLM I like. I just want it to speak."

Vision: Any text model in vLLM can be paired with any TTS decoder as a composable pipeline. The text model generates text; a bridge layer handles sentence segmentation and feeds chunks to the TTS stage via async_chunk.

[Any vLLM text model] -> [sentence chunker] -> [Any TTS decoder] -> audio

This unlocks combinations like Llama + Qwen3-TTS, Mistral + CosyVoice3, or domain-finetuned models + any supported TTS backend. The existing async_chunk framework and WebSocket sentence-boundary logic provide the foundation.

Key design questions:

• How does the bridge stage interact with the config system? Is it a new stage type or a pipeline-level abstraction?
• How do we handle latency? The text model must generate enough text before TTS can start. Async chunk helps but there's a minimum buffering delay.
• How do we route voice/speaker parameters when the text model has no concept of them?

Theme 3: Usability (P0)

TTS models are small enough to run on consumer GPUs. The deployment experience should match that simplicity.

Voice management: The voice upload API, voice cache, and voice clone prompt creation should work as a unified flow. Upload once, use everywhere, persist across restarts.

Zero-config startup: Auto-detect TTS model type from HuggingFace config. Auto-dispatch appropriate stage configs based on model and hardware. Reduce the number of required CLI arguments. Make /v1/audio/speech work out of the box.

Benchmarking: Add vllm bench support for /v1/audio/speech so users can measure TTS performance with the same tooling they use for text models.

Documentation: Complete the TTS model developer guide. Make it easy for community contributors to add new models without deep knowledge of the async_chunk internals.

Theme 4: Duplex and Multi-Turn (P1)

The next generation of speech models (SoulX-Duplug, future duplex models) require persistent sessions, bidirectional streaming, and turn-taking. vLLM-Omni's current request/response model doesn't support this natively.

Approach: Start with request/response integration for models like SoulX-Duplug to validate the model layer. In parallel, draft an architecture RFC for full-duplex support covering session state, concurrent streams, barge-in, and multi-turn context management.

This infrastructure is shared with world model support (#1987), which requires the same multi-turn stateful sessions and bidirectional streaming for robotics and interactive video. As the world model RFC notes, "current streaming is single-turn unidirectional (TTS: text->audio). World Models need a multi-turn loop: client pushes observations, server returns predictions, repeat." The architecture we design here should be general enough to serve both TTS duplex and world model use cases, rather than building separate session management systems.

Theme 5: Reliability & Quality (P1)

Quality gates: Whisper transcription roundtrip catches content errors but not audio quality issues. Add perceptual quality metrics (HNR for distortion, UTMOS for naturalness, speaker embedding similarity for voice cloning) to CI so regressions are caught automatically.

Stability: Fix known memory leaks in async chunk mode. Validate long-running servers with 1000+ requests. Handle edge cases gracefully (codec errors, malformed reference audio, OOM on small GPUs).

Test coverage: Current tests are heavily concentrated on Qwen3-TTS. We need to expand coverage across all supported TTS models (Fish Speech, CosyVoice3, Voxtral, FunAudioChat). Each model should have at minimum: offline inference tests, online serving tests, and streaming tests. Also expand from L2 to L3/L4 for TTS, and ensure quality regression tests cover voice cloning specifically, not just content accuracy.

Theme 6: Model Coverage (P2)

Continue expanding model support, prioritizing models that bring new capabilities:

• S2S models (FunAudioChat, Covo-Audio-Chat) for end-to-end speech conversation
• Duplex models (SoulX-Duplug) for turn-taking
• DiT-based TTS (F5-TTS) for non-autoregressive generation
• Community requests (Kimi-Audio, TADA, MiMo-V2-TTS, MOSS-TTS, Chatterbox, Ming-omni-tts)

The goal is not to add every model, but to ensure the framework is general enough that new models can be integrated quickly by following the developer guide.

Summary

Theme	Priority	Owner	What Success Looks Like
Streaming as core feature	P0	@Sy0307	All TTS models stream by default, end-to-end streaming with text LLMs
TTS as composable layer	P0	@dzhengAP @linyueqian	Any vLLM text model can speak via config
Usability	P0	@JuanPZuluaga	Zero-config TTS startup, unified voice management, vllm bench for speech
Duplex and multi-turn	P1	@four1er @qibaoyuan	Architecture RFC landed, first duplex model working, shared with world models
Reliability & quality	P1	@JuanPZuluaga	Perceptual quality CI gates, no memory leaks, multi-model test coverage
Model coverage	P2	@zhangj1an	3-5 new models, framework generalizes cleanly

Call for Owners

Themes 3-6 need owners. If you're interested in leading or contributing to any of these areas, please comment below:

• Theme 3: Usability - Zero-config startup, voice management unification, vllm bench for speech, documentation.
• Theme 4: Duplex and multi-turn - Design the session management architecture shared with world models. Good fit for someone familiar with WebSocket infrastructure and stateful serving.
• Theme 5: Reliability & quality - Quality metrics CI integration (HNR, UTMOS, speaker similarity), memory leak fixes, L3/L4 test expansion, multi-model test coverage.
• Theme 6: Model coverage - Pick a model and integrate it. Follow the developer guide, contribute back improvements to the framework.

Also looking for feedback on:

• Are we missing any important themes?
• Do the priorities make sense?
• What would you like to see in TTS that isn't listed here?

CC List

@hsliuustc0106 @gcanlin @Sy0307 @JuanPZuluaga @lishunyang12 @Gaohan123 @nemoramo @R2-Y @dubin555 @DomBrown @evezhier @thrashingstate @divyanshsinghvi @anurag12-webster @tzhouam @NickCao @y123456y78 @indevn @sc-hua @TKONIY @yenuo26 @Shirley125 @qibaoyuan @four1er @hongyi-zhang @princepride @patrickvonplaten @ekagra-ranjan @zhangj1an