[RFC]: TTS Development Roadmap - March 2026

Motivation.

Issue #938 tracked Qwen3-TTS to production readiness through February. All original production criteria were met - RTF 0.34 (target <1.0), TTFP 131ms (target <200ms), stable 2-stage deployment, streaming output, and benchmark tooling.

March shifts focus to four themes:

1. Harden Qwen3-TTS - close remaining bugs, land in-progress features, continue performance work
2. Expand TTS model coverage - Fish Speech S2 Pro, GLM-TTS, MOSS-TTS, Ming-omni-tts, Index-TTS, and others
3. Build a TTS integration agent - automate the new-model onboarding process to accelerate model support
4. Improve infrastructure - config refactor, metrics, WebSocket streaming, batch endpoint

This RFC supersedes #938 and serves as the single tracking issue for all TTS work in March.

Proposed Change.

1. Where We Are (Mid-March Summary)

Qwen3-TTS is production-ready. Key merges since #938:

• Performance: Re-prefill code predictor + Code2Wav CUDA graph (#1617), batch .tolist() to eliminate GPU syncs (#1614), flexible initial phase for TTFA reduction (#1583), GPU-resident buffers to eliminate CPU round-trips (#1852)
• Reliability: Fix all-silence output (#1664), fix Base ICL voice clone (#1554, #1731), fix voice clone streaming quality and stop-token crash (#1945), request validation (#1641), HTTP error responses (#1687)
• Infrastructure: Non-async chunk support (#1678), TTS model developer guide (#1693), CI model pre-download (#1727), nightly performance benchmark (#1700), benchmark scripts (#1573), WebSocket streaming (#1719), Gradio demo (#1231), WAV streaming format (#1819), entrypoint refactoring (#1908), config refactor phase 1 (#1115)

Current performance (H100/H200):

Concurrency	TTFP	RTF	Throughput
1	131ms	0.34	2.95x realtime
4	200ms (median)	0.49	7.38x realtime

---

2. Open PRs

2.1 Qwen3-TTS

PR	Title	Author	Priority	Status
#1700	Nightly performance benchmark	@linyueqian	P0	Merged
#1573	Benchmark scripts	@linyueqian	P1	Merged
#1645	Report supported tasks for omni models	@linyueqian	P1	Merged
#1719	WebSocket streaming audio output	@Sy0307	P0	Merged
#1714	Dynamic TTFA based on Code2Wav load	@JuanPZuluaga	P1	Merged
#1852	GPU-resident buffers, boost throughput+latency	@JuanPZuluaga	P1	Merged
#1945	Fix Base voice clone streaming quality and stop-token crash	@linyueqian	P0	Merged
#1985	Keep audio_codes on GPU, eliminate per-step sync stalls	@DomBrown	P1	Under review
#1925	Talker CUDA graph	@evezhier	P1	Under review
#1913	CodePredictor torch.compile with reduce-overhead	@JuanPZuluaga	P1	Under review
#1911	Add Qwen-TTS test cases and unify style	@R2-Y	P1	Under review
#1797	Code2Wav triton SnakeBeta + CUDA graph	@JuanPZuluaga	P1	In progress
#1883	Streaming text input for Qwen3-TTS	@thrashingstate	P1	In progress
#1963	Support changing speaker of Qwen3-Omni	@R2-Y	P2	In progress
#1857	Better Qwen3-TTS online serving demo	@linyueqian	P2	In progress
#1720	Audio/speech metrics (Prometheus)	@Temirulan	P1	In progress
#1230	Streaming text input via WebSocket	@lishunyang12	P1	In progress
#1227	Speaker embedding passthrough API	@marksverdhei	P2	Open
#1389	Base64 input for ref-audio	@lolyhop	P2	Open

2.2 Infrastructure & Cross-Model

PR	Title	Author	Priority	Status
#1115	Model Pipeline Configuration System	@lishunyang12	P0	Merged
#1958	Auto gpu_memory_utilization config tool	@linyueqian	P1	Under review
#1701	Speech batch entrypoint	@divyanshsinghvi	P1	WIP
#1829	Standardize data entry key names	@divyanshsinghvi	P1	Under review
#1851	Remove duplicated config keyword max batch size	@tzhouam	P2	Under review
#1807	RFC: Unify test case stage config YAMLs	-	P2	Open

2.3 New TTS Models

PR	Title	Author	Priority	Status
#1798	Fish Speech S2 Pro support	@linyueqian	P1	Merged
#1859	Fish Speech S2 Pro perf improvement	@Sy0307	P1	Under review
#1803	Mistral TTS support	@y123456y78	P1	In progress
#1370	MOSS-TTS support	@anurag12-webster	P1	In progress
#1517	Chatterbox TTS (Turbo) support	@lishunyang12	P2	In progress
#1748	FunAudioChat S2S support	@nemoramo	P1	Under review
#464	Step-Audio-R1.1 support	@wuli666	P2	WIP
#334	Index-TTS support	@divyanshsinghvi	P2	WIP (since Dec)
#1703	CosyVoice3 async_chunk hardening	@indevn	P1	Open

2.4 Performance & Optimization

PR	Title	Author	Priority	Status
#1985	GPU-resident audio_codes, deferred D2H	@DomBrown	P1	Under review
#1925	Talker CUDA graph	@evezhier	P1	Under review
#1913	CodePredictor torch.compile	@JuanPZuluaga	P1	Under review
#1797	Code2Wav triton SnakeBeta + CUDA graph	@JuanPZuluaga	P1	In progress
#1758	Qwen3-Omni code predictor re-prefill + SDPA	@LJH-LBJ	P1	In progress

---

3. Open Issues

3.1 Bugs

Issue	Title	Severity	Status
#1774	v0.17.0rc1 voice clone produces white noise (Base model)	High	Fixed (#1945)
#1525	Generations hang indefinitely after first few requests	High	Fixed
#1707	Noise insertion at front of voice output	Medium	Fixed
#1477	Stage-1 crash: Prometheus counter negative token count	Medium	Fixed
#1411	Offline Qwen3-TTS generates identical outputs	Medium	Fixed
#1589	qwen3-tts-base ref_text param not working	Medium	Fixed
#1635	Streaming speed too slow (user report)	Medium	Fixed
#1754	v0.16.0 voice cloning bad output	Medium	Fixed
#1061	CUDA Graph not enabled on 5090	Low	Fixed
#1965	Online service crashed	High	Open
#1896	Qwen3-TTS slow on A100 with v0.16.0	Medium	Open
#1849	Qwen3-TTS fails to start on Ascend 910B/C in v0.16.0	Medium	Open

3.2 Feature Requests

Issue	Title	Priority	Status
#1766	Streaming text input for Qwen3-TTS	P1	In progress (#1230, #1883)
#1760	Separate endpoint for voice clone prompt creation	P2	Open
#1676	Stream wav format from speech API	P2	Fixed (#1819)
#1603	Voice clone RFC for Qwen3-TTS Base	P2	Open

3.3 New Model Requests

Issue	Model	PR	Status
#821	GLM-TTS	#834 (closed)	Assigned to @YeungYathin
#1319	MOSS-TTS	#1370	In progress
#899	Chatterbox TTS (Turbo)	#1517	In progress (reopened)
#1551	F5-TTS	-	Open
#229, #545	Index-TTS / Index-TTS2	#334 (WIP)	WIP
#1461	Ming-omni-tts	-	Open
#1154	VoxCPM	-	Open
#184	VibeVoice	-	Open
#862	Step-Audio-R1.1	#464 (WIP)	WIP
#271, #272	Step Audio 2 / Step Audio EditX	-	Open
#1824	Kimi-Audio-7B	-	New
#1962	VoXtream2	-	New
#1967	SoulX-Duplug-0.6B	-	New
#1748	FunAudioChat S2S	#1748	In progress

---

4. March Work Plan

4.1 P0: Must Ship

Task	PR/Issue	Owner	ETA	Status
Nightly performance benchmark CI	#1700	@linyueqian	Mar 10	Done
WebSocket streaming audio	#1719	@Sy0307	Mar 16	Done
Fix hanging requests	#1525	@linyueqian	Mar 16	Done
Fix v0.17 voice clone regression	#1774	@Sy0307	Mar 10	Done (#1945)
Config refactor phase 1	#1115	@lishunyang12	Mar 23	Done (Mar 12)

4.2 P1: Should Ship

Task	PR/Issue	Owner	ETA	Status
Audio/speech Prometheus metrics	#1720	@Temirulan	Mar 16	In progress
Dynamic TTFA	#1714	@JuanPZuluaga	Mar 16	Done
Streaming text input	#1230, #1766, #1883	@lishunyang12, @thrashingstate	Mar 23	In progress
Speech batch endpoint	#1701	@divyanshsinghvi	Mar 23	WIP
CosyVoice3 stabilization	#1685, #1703	@divyanshsinghvi, @indevn	Mar 23	#1685 Done, #1703 open
Fish Speech S2 Pro	#1798	@linyueqian	Mar 16	Done
Fish Speech S2 Pro perf	#1859	@Sy0307	Mar 23	Under review
MOSS-TTS integration	#1370	@anurag12-webster	Mar 30	In progress
TTS integration agent	-	@linyueqian	Ongoing	See section 5.2
Qwen3-TTS test coverage	#1911	@R2-Y	Mar 23	Under review
Performance: talker CUDA graph	#1925	@evezhier	Mar 23	Under review
Performance: CodePredictor torch.compile	#1913	@JuanPZuluaga	Mar 23	Under review
Performance: GPU-resident audio_codes	#1985	@DomBrown	Mar 23	Under review

4.3 P2: Stretch Goals / Start This Month

Task	Issue	Owner	Notes
GLM-TTS	#821	@YeungYathin	AR -> Vocoder, 2-stage. Previous PR #834 closed. Newly assigned. Follow #1225 and #1693.
Ming-omni-tts	#1461	Unassigned	inclusionAI/Ming-omni-tts-16.8B-A3B. AR+FlowMatching -> AudioVAE, 2-stage.
Index-TTS	#334	@divyanshsinghvi	WIP since Dec, needs rebase.
F5-TTS	#1551	@ekagra-ranjan	Flow-matching DiT. May need new stage types.
Chatterbox TTS (Turbo)	#899	@lishunyang12	PR #1517 reopened, in progress.
FunAudioChat S2S	#1748	@nemoramo	AR -> Code2Wav, 2-stage. Under review.
Step-Audio-R1.1	#862	@wuli666	PR #464 WIP.
Kimi-Audio-7B	#1824	Unassigned	New model request.
VoXtream2	#1962	Unassigned	New model request.
SoulX-Duplug-0.6B	#1967	Unassigned	New model request.
Speaker embedding passthrough	#1227	@marksverdhei	Pre-computed x-vectors, voice blending.

---

5. Architecture & Design

5.1 New Model Onboarding (Manual Path)

Per RFC #1225 and developer guide #1693:

1. Define stage architecture (AR, decoder, vocoder, etc.)
2. Create YAML stage config with async_chunk: true
3. Implement stage input processor (<stage_a>2<stage_b>_async_chunk)
4. Add chunked_decode_streaming() to final stage
5. Add e2e tests and examples

5.2 TTS Integration Agent (Automated Path)

Build an agent to automate new TTS model onboarding. Given a HuggingFace model, the agent should:

• Analyze model architecture (AR, DiT, flow-matching, etc.) and identify stage boundaries
• Generate YAML stage config following patterns in RFC #1225
• Scaffold stage input processors, model wrappers, and e2e tests
• Validate against the TTS model developer guide (#1693)

This reduces manual effort per model and enables the community to contribute models faster. Especially valuable for AR-based models that follow the same pattern as Qwen3-TTS.

Owner: @linyueqian

5.3 WebSocket vs HTTP Streaming

Both should coexist:

• HTTP /v1/audio/speech with Transfer-Encoding: chunked for OpenAI-compatible clients
• WebSocket /v1/audio/speech/ws for interactive/bidirectional use cases
• WebSocket enables streaming text input (#1230)

5.4 Metrics Strategy (#1720)

Key metrics to standardize across all TTS models:

• tts_request_duration_seconds (histogram)
• tts_time_to_first_audio_seconds (histogram)
• tts_realtime_factor (histogram)
• tts_active_requests (gauge)
• tts_tokens_per_second (summary)

5.5 Config Refactor Impact (#1115)

PR #1115 has merged (Mar 12). All existing TTS configs will need migration to the new format. New models should target the new format.

---

6. Model Support Matrix

Model	Architecture	Stages	Offline	Online	Streaming	async_chunk	Status
Qwen3-TTS	AR -> SpeechTokenizer	2	Yes	Yes	Yes	Yes	Production
Fish Speech S2 Pro	Dual-AR (Slow+Fast) -> DAC	3	Yes	Yes	Yes	Yes	Merged
CosyVoice3	AR -> Flow+HiFiGAN	2	Yes	Yes	Partial	WIP	Beta
Mistral TTS	AR -> Mimi Decoder	2	WIP	-	-	-	In Progress
MOSS-TTS	AR -> AudioDecoder	2	WIP	-	-	-	In Progress
Index-TTS	GPT+DVAE -> Vocoder	2	WIP	-	-	-	WIP
GLM-TTS	AR -> Vocoder	2	-	-	-	-	Not Started
Ming-omni-tts	AR+Flow -> AudioVAE	2	-	-	-	-	Not Started
F5-TTS	DiT (flow-matching)	1-2	-	-	-	-	Not Started
Chatterbox TTS (Turbo)	-	-	-	-	-	-	In Progress (#1517)
FunAudioChat S2S	AR -> Code2Wav	2	WIP	-	-	-	In Progress (#1748)
Step-Audio-R1.1	TBD	TBD	WIP	-	-	-	WIP (#464)
VoxCPM	AR<->DIT -> Vocoder	2	-	-	-	-	Not Started
VibeVoice	TBD	TBD	-	-	-	-	Not Started
Kimi-Audio-7B	TBD	TBD	-	-	-	-	New Request
VoXtream2	TBD	TBD	-	-	-	-	New Request
SoulX-Duplug-0.6B	TBD	TBD	-	-	-	-	New Request

---

7. Timeline

Week	Focus
Mar 10-14	Fix #1774 (voice clone regression), land #1700 (nightly benchmark), land #1645 All done
Mar 16-21	Land #1719 (WebSocket streaming), #1720 (metrics), #1714 (dynamic TTFA), investigate #1525 #1719, #1714, #1525 done; #1720 still in progress
Mar 23-28	#1701 (batch endpoint), #1230/#1883 (streaming input), CosyVoice3 #1703, performance PRs (#1925, #1913, #1985, #1797), test coverage (#1911)
Mar 30+	MOSS-TTS integration, new model PRs (Mistral TTS, GLM-TTS, Ming-omni-tts), TTS agent iteration

Feedback Period.

Two weeks (Mar 10 - Mar 24, 2026).

CC List.

@hsliuustc0106 @Gaohan123 @gcanlin @Sy0307 @linyueqian @JuanPZuluaga @divyanshsinghvi @lishunyang12 @Temirulan @ekagra-ranjan @marksverdhei @anurag12-webster @indevn @ZhanqiuHu @Lidang-Jiang @LJH-LBJ @gerayking @tsdocode @xulusjb @dubin555 @zhaotyer @RomanKoshkin @YanickSchraner @JackLeeHal @yuanheng-zhao @lolyhop @pengchengneo @farzanehnakhaee70 @MyronWang @qibaoyuan @wuli666 @nemoramo @lengrongfu @Etelis @ashok-arora @david6666666 @DomBrown @evezhier @thrashingstate @R2-Y @tzhouam

Any Other Things.

Related Issues & RFCs:

• #938 - Qwen3-TTS Production Ready (February milestone, predecessor)
• #1225 - General TTS Model Implementation (architecture patterns)
• #1479 - WebSocket Streaming Audio Output RFC (completed)
• #1595 - Decouple Model-Specific Logic from Model Runner
• #1600 - TTS Model Performance Optimization on NPU
• #886 - NPU 2026 Q1 Roadmap
• #1131 - Modular Chunk-Based Communication Architecture
• #1770 - Migrate to Model Runner V2
• #1603 - Voice Clone RFC for Qwen3-TTS Base
• #808 - vLLM-Omni Model Support (master tracking)
• #1807 - Unify Migration of Omni Test Case Stage Config YAMLs (new)