[RFC]: Benchmark Layout Reorganization

Motivation.

The current layout has several issues:

1. It is hard to answer basic questions such as "where are all TTS serving benchmarks?"
2. Shared utilities are hidden behind one model's directory naming.
3. New benchmark contributors have no clear pattern to follow.
4. Cross-model comparisons within the same modality are awkward.
5. Documentation and scripts drift because each folder evolves independently.

We want a layout that optimizes for:

• modality-first discoverability
• reusable shared utilities
• clear separation between framework code, model-specific configs, and outputs
• easy onboarding for new benchmark contributors
• consistent documentation and result reporting

Proposed Change.

Goals

• Group benchmark suites by modality first: tts, omni, diffusion
• Preserve room for model-specific benchmark logic under each modality
• Standardize where scripts, configs, datasets, results, and docs live
• Make it straightforward to add a new model benchmark by copying an existing pattern
• Reduce one-off benchmark helper placement such as fish-speech/fish_bench_utils.py

Non-Goals

• This RFC does not redesign benchmark metrics themselves
• This RFC does not merge all benchmark clients into a single implementation
• This RFC does not change accuracy/ or distributed/ in the first step
• This RFC does not require renaming every benchmark in one large PR

Feedback Period.

No response

CC List.

cc @linyueqian @ZeldaHuang @Sy0307 @amy-why-3459

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.