| --- |
| title: Conference Generator VibeVoice |
| emoji: ⭐ |
| colorFrom: indigo |
| colorTo: red |
| sdk: gradio |
| sdk_version: "5.44.1" |
| app_file: app.py |
| pinned: false |
| --- |
| |
| <p align="center"> |
| <img src="public/images/banner.png" alt="VibeVoice Conference Generator" width="100%"/> |
| </p> |
|
|
| # Conference Generator — powered by VibeVoice |
|
|
| Generate realistic multi-speaker conference calls, meetings, and podcasts from a single text prompt. The app uses an LLM to write a natural-sounding script, then synthesizes long-form multi-speaker audio with Microsoft's [VibeVoice](https://huggingface.co/microsoft/VibeVoice-1.5B) model. |
|
|
| **Try it live:** [Hugging Face Space](https://huggingface.co/spaces/ACloudCenter/Conference-Generator-VibeVoice) |
|
|
| ### Listen to the Demo |
|
|
| https://github.com/user-attachments/assets/cfe5397f-7aad-4662-b0b8-e62d7546a9fb |
|
|
| _A 3-speaker example — Wizard (Chicago), Orc (Janus), and Mom (Cherry) — generated from a single sentence prompt. Audio visualizer created separately._ |
|
|
| --- |
|
|
| ## Features |
|
|
| - **Prompt-to-audio in one step** — describe the scenario ("a 4-person product meeting about pricing") and get a full generated conversation |
| - **1–4 speakers** with 6 distinct voice presets (male/female tagged) |
| - **Up to ~90 minutes of continuous speech** via VibeVoice's long-form generation |
| - **Editable turn-by-turn script** — tweak speaker assignments or dialogue before rendering |
| - **Title generation** — the LLM names each script automatically |
| - **Two model sizes** — VibeVoice-1.5B (fast) and VibeVoice-7B (higher quality) |
| - **Gender-aware voice casting** — female characters get female voices automatically (Mom → Cherry, Wizard → Chicago, etc.) with one-click override |
| - **Voice preview** — sample any of the 6 voices before committing to a long generation |
|
|
| --- |
|
|
| ## Walkthrough |
|
|
| ### 1. Describe your scenario |
|
|
| Type any scenario — a meeting, podcast, argument, TED talk — and the LLM writes the full script. |
|
|
| <p align="center"> |
| <img src="public/images/Screenshot-1.png" alt="Step 1: Prompt input" width="100%"/> |
| </p> |
|
|
| ### 2. Review the script and pick voices |
|
|
| Speaker tags auto-assign by gender. Every voice dropdown stays in sync with the tags above. Preview any voice before generating. |
|
|
| <p align="center"> |
| <img src="public/images/Screenshot-2.png" alt="Step 2: Script editor with voice sync" width="100%"/> |
| </p> |
|
|
| ### 3. Generate the audio |
|
|
| Kick off the GPU job on Modal. A funny parody narration keeps you entertained during the wait. |
|
|
| <p align="center"> |
| <img src="public/images/Screenshot-3.png" alt="Step 3: Generating" width="85%"/> |
| </p> |
|
|
| ### 4. Listen and download |
|
|
| Full-length multi-speaker audio, ready to play or download as a WAV. |
|
|
| <p align="center"> |
| <img src="public/images/Screenshot-4.png" alt="Step 4: Complete" width="85%"/> |
| </p> |
|
|
| --- |
|
|
| ## About VibeVoice |
|
|
| VibeVoice is Microsoft's open-source long-form, multi-speaker TTS model. It uses a frozen LLM backbone with acoustic + semantic tokenizers and a diffusion head to produce up to 90 minutes of natural conversational audio with up to 4 distinct speakers. |
|
|
| <p align="center"> |
| <img src="public/images/diagram.jpg" alt="VibeVoice architecture" width="85%"/> |
| </p> |
|
|
| <p align="center"> |
| <em>Speaker voice prompts and a plain text script are fed into the VibeVoice backbone, which streams audio chunks through per-turn diffusion heads.</em> |
| </p> |
|
|
| ### Benchmark performance |
|
|
| <p align="center"> |
| <img src="public/images/chart.png" alt="VibeVoice benchmark comparison" width="75%"/> |
| </p> |
|
|
| VibeVoice leads on preference, realism, and richness among long-form multi-speaker TTS models. |
|
|
| --- |
|
|
| ## Architecture |
|
|
| This project separates the lightweight Gradio frontend (hosted on HF Spaces) from the GPU-heavy model backend (hosted on [Modal](https://modal.com)). |
|
|
| ``` |
| ┌──────────────────────┐ ┌─────────────────────────┐ |
| │ HF Space (Gradio) │ │ Modal (GPU backend) │ |
| │ ───────────────── │ │ ─────────────────── │ |
| │ • Prompt UI │ ───► │ • VibeVoice-1.5B / 7B │ |
| │ • Script editor │ │ • Voice prompt loader │ |
| │ • Qwen2.5-Coder 32B │ │ • Long-form synthesis │ |
| │ (script writing) │ ◄─── │ • Returns WAV bytes │ |
| └──────────────────────┘ └─────────────────────────┘ |
| ``` |
|
|
| - **Frontend** (`app.py`): Gradio UI, script generation via HF Inference API (Qwen2.5-Coder-32B), script parsing, playback. |
| - **Backend** (`backend_modal/`, not included in this repo): deployed separately on Modal as a class-based GPU service exposing `generate_podcast`. |
|
|
| --- |
|
|
| ## Voices |
|
|
| | Voice | Gender | |
| | --------- | :----: | |
| | Cherry | F | |
| | Chicago | M | |
| | Janus | M | |
| | Mantis | F | |
| | Sponge | M | |
| | Starchild | F | |
|
|
| Voice samples live in `public/voices/` and are loaded as short reference clips by the VibeVoice backend. |
|
|
| --- |
|
|
| ## Running locally |
|
|
| ```bash |
| git clone https://github.com/Josh-E-S/vibevoice-conference-generator.git |
| cd vibevoice-conference-generator |
| pip install -r requirements.txt |
| |
| # Set your HF token (used for the script-writing LLM) |
| export HF_TOKEN=your_hf_token_here |
| |
| # Deploy the Modal backend separately (not in this repo) |
| # modal deploy backend_modal/modal_runner.py |
| |
| python app.py |
| ``` |
|
|
| Required env: |
|
|
| - `HF_TOKEN` — Hugging Face token with Inference API access |
|
|
| --- |
|
|
| ## Repo layout |
|
|
| ``` |
| . |
| ├── app.py # Gradio frontend + script generation |
| ├── requirements.txt # gradio, modal, huggingface_hub |
| ├── public/ |
| │ ├── images/ # Banner, architecture diagram, screenshots |
| │ ├── voices/ # Voice reference clips (Cherry, Chicago, ...) |
| │ └── sample-generations/ # Example generations |
| ├── text_examples/ # Example scripts (1p, 2p, 3p, 4p scenarios) |
| ├── tests/ # Parser tests + example prompts |
| └── README.md |
| ``` |
|
|
| --- |
|
|
| ## Credits |
|
|
| - **[VibeVoice](https://github.com/microsoft/VibeVoice)** — Microsoft Research's long-form multi-speaker TTS model |
| - **[Qwen2.5-Coder-32B](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct)** — script generation |
| - **[Modal](https://modal.com)** — GPU compute for inference |
| - **[Gradio](https://gradio.app)** + **[Hugging Face Spaces](https://huggingface.co/spaces)** — frontend hosting |
|
|
| --- |
|
|
