[New Model]: dots.tts (rednote-hilab) — continuous-AR 48kHz zero-shot voice cloning

The model to consider.

dots.tts (rednote-hilab / RedNote-hilab AI team) — a fully continuous, end-to-end autoregressive zero-shot voice-cloning TTS model.

• Code: https://github.com/rednote-hilab/dots.tts
• Models: https://huggingface.co/collections/rednote-hilab/dotstts
• Demo (online): https://huggingface.co/spaces/rednote-hilab/dots.tts
• Demo (page): https://rednote-hilab.github.io/dots.tts-demo/

Architecture (~2B params):

• AR backbone: Qwen2.5-1.5B-Base LLM
• Acoustic head: autoregressive flow-matching head (DiTAR / ARDiT-style continuous-AR design)
• Codec: a 48 kHz AudioVAE that encodes waveforms into continuous latents — there are no discrete audio tokens anywhere in the pipeline
• Speaker encoder: CAM++ x-vector for zero-shot cloning
• Vocoder: BigVGAN-style

Variants (all 2B, same arch, different post-training):

• rednote-hilab/dots.tts-soar — aligned/default checkpoint, highest speaker similarity → recommended for voice cloning
• rednote-hilab/dots.tts-base — pretrained base, intended for fine-tuning your own voice
• rednote-hilab/dots.tts-mf — MeanFlow-distilled, 2–4 step generation for real-time / low-latency

Specs & results:

• 48 kHz mono hi-fi output, 24 languages (zh / en / ja / ko / fr / de, etc.)
• Seed-TTS-Eval (Chinese): CER 0.94%, speaker similarity 81.0 — top tier among open-source
• Multilingual: average speaker similarity 83.9 across 24 languages
• License: Apache-2.0 (free commercial use)

The closest model vllm-omni already supports.

VoxCPM2 — both are continuous-latent, flow-matching TTS models rather than discrete-codec/token AR models. Key differences:

• dots.tts uses a Qwen2.5-1.5B AR backbone + an AR flow-matching acoustic head over continuous 48 kHz AudioVAE latents (no discrete codebook decode stage), similar in spirit to VoxCPM2's continuous VAE + flow-matching design.
• Output is 48 kHz (vs. the more common 24 kHz in the current stack), so the decode/VAE stage and any audio-postprocessing assumptions need to handle the higher sample rate.
• The mf MeanFlow-distilled variant collapses generation to 2–4 steps, which maps well onto a fast Stage-1 decode path.

What's your difficulty of supporting the model you want?

• Continuous-AR, no discrete tokens: there is no codebook/codec-token stage to schedule — the AR step emits continuous latents through a flow-matching head, so it fits the VoxCPM2-style continuous pipeline more than the Qwen3-TTS talker + code-predictor pattern. Stage separation would be (LLM AR) → (flow-matching acoustic head + AudioVAE/BigVGAN decode).
• 48 kHz path: AudioVAE decode and BigVGAN vocoder run at 48 kHz; streaming chunking / holdback and any resampling assumptions in the audio output pipeline need to account for this.
• Speaker encoder preprocessing: CAM++ x-vector extraction on the reference audio runs as a preprocessing step (like other zero-shot cloning models in the repo).
• Flow-matching sampling steps: soar/base use multi-step flow matching; mf uses 2–4 step MeanFlow. Batched inference needs to handle the per-request step schedule.
• Trust-remote-code / custom modeling: model ships custom modeling code on the Hub; standard new-model wiring (Stage 0 / Stage 1 separation, streaming, CUDA-graph) applies.

Use case and motivation

• High-fidelity 48 kHz cloning: most open TTS in the stack is 24 kHz; dots.tts brings native 48 kHz output with strong speaker similarity.

• Strong open-source numbers: Chinese CER 0.94% / SIM 81.0 on Seed-TTS-Eval and 83.9 average multilingual similarity put it at the front of open-source voice cloning.

• Real-time variant: the mf MeanFlow 2–4 step checkpoint targets low-latency / streaming serving — a natural fit for /v1/audio/speech.

• Permissive license: Apache-2.0 weights + code allow free commercial use.

• Three checkpoints: soar (clone), base (fine-tune), mf (real-time) cover serving, customization, and latency-sensitive scenarios from one family.

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.