[New Model]: SoulX-Duplug-0.6B (Soul-AILab/SoulX-Duplug-0.6B)

The model to consider.

https://huggingface.co/Soul-AILab/SoulX-Duplug-0.6B

SoulX-Duplug — a 0.6B parameter plug-and-play streaming semantic VAD module for real-time full-duplex speech conversation.

• Architecture: Qwen3-0.6B backbone with GLM-4-Voice speech tokenizer (12.5Hz), LoRA fine-tuned (r=32) for dialogue state prediction
• Paradigm: Interleaved prediction {audio_tokens, ASR_text, state} per 160ms chunk
• States: user_idle, user_nonidle, user_backchannel, user_complete, user_incomplete
• Block-causal: 160ms target window, 960ms look-back, 40ms look-ahead, total 1,160ms receptive field
• Bilingual: English and Chinese
• Latency: ~205-250ms practical (ultra-low for turn-taking)
• VRAM: ~7.8 GB total (base model + LoRA + tokenizer)
• Paper: arXiv:2603.14877 (Interspeech 2026 submission)
• License: Apache 2.0

The closest model vllm-omni already supports.

No direct equivalent in vllm-omni. This is a dialogue state prediction / semantic VAD module, not a TTS or ASR model. However:

• The backbone is Qwen3-0.6B which vllm-omni already supports via the Qwen family
• The speech tokenizer is GLM-4-Voice (12.5Hz codec)
• It's designed as a plug-and-play module that sits in front of a TTS pipeline (e.g., could work with Qwen3-TTS or other supported TTS models)

What's your difficulty of supporting the model you want?

• Speech tokenizer integration: Needs GLM-4-Voice tokenizer (1.46GB) to convert audio chunks to speech tokens at 12.5Hz — this is a preprocessing step before the LLM
• Streaming chunk-based inference: Processes 160ms audio blocks with block-causal attention, requiring stateful inference across chunks
• Hybrid inference: At inference time uses an external ASR model (SenseVoice Small) for teacher-forced text tokens — this is a multi-model orchestration challenge
• Custom vocabulary: Expanded Qwen3 vocab (151,936 tokens) with special state tokens
• LoRA checkpoint: Model is distributed as base + LoRA adapter (3.9GB .pth)
• Not a standard generation task: Output is a classification-like state prediction per chunk, not free-form text or audio generation

Use case and motivation

• Full-duplex voice AI: Enables natural turn-taking in real-time voice conversation systems — knows when the user is done speaking (semantically, not just acoustically) so the system can respond without awkward pauses or interruptions

• Plug-and-play with existing TTS: Designed to sit in front of any TTS pipeline. Combined with vllm-omni's Qwen3-TTS support, this could enable a complete full-duplex voice system served by vllm-omni

• State-of-the-art turn-taking: Turn-Taking TOR 0.933 vs Gemini Live's 0.655, with 0.511s response latency vs 1.301s

• Streaming-native: 160ms chunk processing fits naturally into vllm-omni's streaming architecture

• Lightweight: Only 0.6B params, can run alongside TTS models on the same GPU

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.