[New Model] F5-TTS (flow-matching DiT-based TTS)

Model Information

• Model: F5-TTS v1
• Type: TTS (Text-to-Speech)
• HuggingFace: https://huggingface.co/SWivid/F5-TTS
• Paper/Repo: https://github.com/SWivid/F5-TTS

Why support this model?

F5-TTS is one of the most popular open-source TTS models with 14k+ GitHub stars. It uses a fully non-autoregressive flow-matching approach with a DiT backbone — architecturally similar to diffusion models already in vllm-omni. Its simplicity (no duration model, no phoneme alignment, no separate text encoder) and fast inference (RTF 0.15) make it well-suited for high-throughput serving.

Technical Notes

• DiT architecture with flow matching (non-autoregressive)
• Text is padded with filler tokens to match speech length, then denoised
• ConvNeXt module refines text representation for alignment
• Sway Sampling inference strategy for improved quality/speed
• v1 base model released 2025/03/12 with improved training and inference
• Zero-shot voice cloning with short reference audio
• Standalone implementation (not yet in diffusers); PyPI package f5-tts available

Closest supported model in vllm-omni

Stable Audio Open — also a diffusion-based audio generation model. F5-TTS's DiT + flow matching architecture is also conceptually similar to the image DiT models (FLUX, SD3) already supported, since the core denoising loop follows the same pattern but operates on mel spectrograms instead of pixel latents.