Update README.md

README.md

@@ -1,51 +1,222 @@
 ---
 license: apache-2.0
 language:
-- en
-- zh
+  - en
+  - zh
 pipeline_tag: video-to-video
+tags:
+  - video-editing
+  - video-to-video
+  - streaming-video
+  - real-time-video
+  - autoregressive-diffusion
+  - multimodal
+  - joyai
 ---
-<h1 align="center">JoyAI-Video-Edit<br><sub><sup>Real-Time Open-Ended Video Editing with Autoregressive Diffusion</sup></sub></h1>
 
-<div align="center">
+<h1 align="center">JoyAI-Video-Edit</h1>
 
-[![Arxiv](https://img.shields.io/badge/Report-PDF-red)](https://arxiv.org/abs/2608.03974)
-[![Project](https://img.shields.io/badge/Project-JoyAI--Video--Edit-333399)](https://github.com/jd-opensource/JoyAI-Video-Edit)
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Checkpoint-JoyAI--Video--Edit-yellow)](https://huggingface.co/jdopensource/JoyAI-Video-Edit)&#160;
-[![ModelScope](https://img.shields.io/badge/%F0%9F%A4%96%20ModelScope-JoyAI--Video--Edit-624aff)](https://modelscope.cn/models/jd-opensource/JoyAI-Video-Edit)&#160;
-[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
+<h3 align="center">
+Real-Time Open-Ended Video Editing with Autoregressive Diffusion
+</h3>
 
+<p align="center">
+  <a href="https://arxiv.org/abs/2608.03974">
+    <img src="https://img.shields.io/badge/Paper-arXiv-red" alt="Paper">
+  </a>
+  <a href="https://github.com/jd-opensource/JoyAI-Video-Edit">
+    <img src="https://img.shields.io/badge/Code-GitHub-black" alt="GitHub">
+  </a>
+  <a href="https://huggingface.co/jdopensource/JoyAI-Video-Edit">
+    <img src="https://img.shields.io/badge/%F0%9F%A4%97%20Model-JoyAI--Video--Edit-yellow" alt="Hugging Face">
+  </a>
+  <a href="https://joyai-labs.jd.com/v2v/">
+    <img src="https://img.shields.io/badge/%F0%9F%9A%80%20Demo-Streaming--V2V-orange" alt="Demo">
+  </a>
+  <a href="https://github.com/jd-opensource/JoyAI-Video-Edit/blob/main/LICENSE">
+    <img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License">
+  </a>
+</p>
 
-</div>
+<p align="center">
+  <img
+    src="https://raw.githubusercontent.com/jd-opensource/JoyAI-Video-Edit/main/assets/teaser.jpg"
+    width="96%"
+    alt="JoyAI-Video-Edit teaser"
+  >
+</p>
 
 ## 🐶 JoyAI-Video-Edit
 
-JoyAI-Video-Edit is a real-time, instruction-guided video editing system for open-ended video streams. Given a live camera stream or uploaded video and a natural-language edit instruction, it edits frames causally as they arrive, without waiting for the full video, requiring a predefined video length, or revisiting future frames. In our deployment benchmark, the full end-to-end pipeline reaches 30.19 FPS at 720x1280, pushing video editing from offline batch processing toward interactive streaming generation.
+**JoyAI-Video-Edit** is a real-time, instruction-guided video editing framework designed for open-ended video streams.
 
-The system combines an MLLM-based condition encoder, a causal video VAE, and a 16B-parameter multimodal diffusion transformer. It is trained and deployed as an autoregressive diffusion editor, then accelerated with aligned autoregressive distribution matching distillation, long-horizon optimization, bounded KV-state inference, and deployment-oriented scheduling to sustain high-throughput 720p editing while reducing train-inference mismatch and accumulated temporal drift.
+Unlike conventional offline video editing models that require a complete video before inference, JoyAI-Video-Edit processes frames **causally as they arrive**. It can continuously edit a live camera stream or an uploaded video according to natural-language instructions without accessing future frames or requiring a predefined video length.
+
+The framework combines:
+
+- an **MLLM-based condition encoder** for understanding editing instructions,
+- a **causal video VAE** for streaming visual representation,
+- and a **16B multimodal diffusion transformer** for high-quality video editing.
+
+Through autoregressive diffusion modeling, aligned autoregressive distribution matching distillation, long-horizon optimization, bounded KV-state inference, and deployment-oriented scheduling, JoyAI-Video-Edit achieves stable long-range editing while substantially reducing inference latency.
+
+Our deployment system reaches **30.19 FPS end-to-end at 720 × 1280 resolution**, enabling real-time streaming video editing.
+
+---
+
+## 🔥 News
+
+- **2026.08.06** — 🎉 We released the live demo of JoyAI-Video-Edit.  
+  Try it here: **[Streaming V2V Demo](https://joyai-labs.jd.com/v2v/)**
+
+- **2026.08.05** — 🎉 We released the model checkpoints, deployment code, online demo, and technical report.
+
+---
 
 ## 💎 Highlights
 
-- **Real-time open-ended editing.** Edits live or uploaded videos as frames arrive, without requiring the full sequence upfront.
-- **Diverse instruction control.** Supports subject edits, local edits, background changes, style transfer, motion changes, and reference-guided editing.
-- **Autoregressive diffusion design.** Combines an MLLM condition encoder, causal video VAE, and MMDiT backbone for streaming video editing.
-- **High-throughput 720p deployment.** Reaches 30.19 FPS end-to-end throughput at 720x1280 with bounded KV-state inference and stable per-chunk compute.
+### ⚡ Real-Time Streaming Editing
+
+JoyAI-Video-Edit edits frames continuously as they arrive instead of waiting for the entire video.
+
+It supports open-ended streams and does not require a predefined video duration.
+
+### 🎯 Instruction-Guided Video Editing
+
+The model supports a broad range of editing instructions, including:
+
+- Subject modification
+- Subject addition
+- Subject removal
+- Subject replacement
+- Local object editing
+- Background replacement
+- Style transfer
+- Appearance modification
+- Motion editing
+- Reference-guided video editing
+
+### 🔄 Autoregressive Diffusion
+
+JoyAI-Video-Edit introduces an autoregressive diffusion framework consisting of:
+
+- MLLM condition encoder
+- Causal video VAE
+- 16B multimodal diffusion transformer
+- Chunk-wise autoregressive generation
+- Long-horizon temporal optimization
+- Bounded KV-state inference
+
+### 🚀 Real-Time 720p Deployment
+
+The full deployment pipeline achieves:
+
+**30.19 FPS @ 720 × 1280**
+
+with end-to-end streaming inference.
+
+---
+
+## 🎬 Demo
+
+Try our online real-time video editing demo:
+
+👉 **https://joyai-labs.jd.com/v2v/**
+
+Project repository:
+
+👉 **https://github.com/jd-opensource/JoyAI-Video-Edit**
+
+Technical report:
+
+👉 **https://arxiv.org/abs/2608.03974**
+
+---
+
+## 📦 Model Download
+
+The released JoyAI-Video-Edit checkpoints are hosted on Hugging Face.
+
+### Hugging Face CLI
+
+First install the Hugging Face CLI:
+
+```bash
+pip install -U huggingface_hub
+```
+
+Then download the complete model repository:
+
+```bash
+hf download jdopensource/JoyAI-Video-Edit \
+    --local-dir ./deploy/deps/checkpoints/JoyAI-Video-Edit
+```
+
+Alternatively, you can use:
+
+```bash
+huggingface-cli download jdopensource/JoyAI-Video-Edit \
+    --local-dir ./deploy/deps/checkpoints/JoyAI-Video-Edit
+```
+
+The downloaded checkpoint directory should look like:
+
+```text
+deploy/deps/checkpoints/JoyAI-Video-Edit/
+├── config.json
+├── dit/
+│   └── joyai_video_edit_dit_0804.pth
+└── vae/
+    ├── config.json
+    └── diffusion_pytorch_model.safetensors
+```
+
+> **Note**
+>
+> JoyAI-Video-Edit currently uses its own deployment and inference pipeline.
+> The released checkpoint should **not** be loaded directly with
+> `diffusers.DiffusionPipeline.from_pretrained()`.
+
+MiMo-VL and the ONNX detector files are additional runtime dependencies. Please refer to the deployment documentation in the GitHub repository for details.
+
+---
 
 ## 🚀 Quick Start
 
-### 1. Install
+### 1. Clone the Repository
+
+```bash
+git clone https://github.com/jd-opensource/JoyAI-Video-Edit.git
+cd JoyAI-Video-Edit
+```
+
+### 2. Create the Environment
 
 ```bash
 conda create -n joyai-video-edit python=3.10 -y
 conda activate joyai-video-edit
+
 python -m pip install -r requirements.txt
 ```
 
-### 2. Prepare Checkpoints
+### 3. Download JoyAI-Video-Edit
+
+```bash
+hf download jdopensource/JoyAI-Video-Edit \
+    --local-dir ./deploy/deps/checkpoints/JoyAI-Video-Edit
+```
+
+### 4. Prepare Runtime Dependencies
+
+MiMo-VL and the required ONNX detector files are external runtime dependencies.
+
+Please follow:
 
-Download the released weights from the Hugging Face link above. MiMo-VL and the ONNX detector files are external runtime dependencies; see [`DEPLOYMENT.md`](DEPLOYMENT.md) for deployment details.
+**[DEPLOYMENT.md](https://github.com/jd-opensource/JoyAI-Video-Edit/blob/main/DEPLOYMENT.md)**
 
-### 3. Launch
+for detailed checkpoint preparation and deployment instructions.
+
+### 5. Launch
 
 ```bash
 cd deploy
@@ -58,25 +229,67 @@ Then open:
 http://localhost:8080
 ```
 
-For remote machines, bind the server to `0.0.0.0` and open the selected port, or use SSH port forwarding.
+For deployment on a remote server, bind the service to `0.0.0.0` and expose the selected port or use SSH port forwarding.
 
+---
 
-## 📦 Model Download
+## 🛠️ Custom Deployment
 
-Download the released JoyAI-Video-Edit weights from [Hugging Face](https://huggingface.co/jdopensource/JoyAI-Video-Edit), then place them under:
+For custom deployment settings, edit:
 
 ```text
-deploy/deps/checkpoints/JoyAI-Video-Edit/
-|-- dit/
-|   `-- joyai_video_edit_dit_0804.pth
-`-- vae/
-    |-- config.json
-    `-- diffusion_pytorch_model.safetensors
+deploy/run_server.sh
 ```
 
-<a id="quick-start"></a>
+You can configure:
+
+- JoyAI-Video-Edit checkpoint paths
+- CUDA device placement
+- Server host
+- Server port
+- Runtime cache directories
+
+The default deployment script uses persistent TorchInductor, Triton, and CUDA cache directories so that compiled artifacts can be reused across launches.
+
+`TORCHINDUCTOR_AUTOGRAD_CACHE` is not required for inference-only serving.
+
+---
+
+## 🚧 Roadmap
+
+- [ ] **Consumer GPU support**  
+  Optimize deployment for consumer GPUs such as NVIDIA GeForce RTX 5090.
+
+- [ ] **Stronger JoyAI-Video-Edit model**  
+  Improve overall editing quality with a particular focus on reference-image-guided video editing (RV2V).
+
+- [ ] **Training framework**  
+  Release the complete training pipeline.
+
+- [ ] **Data pipeline**  
+  Release the data generation and processing framework.
+
+---
+
+## 📚 Citation
+
+If JoyAI-Video-Edit is useful for your research or project, please cite:
+
+```bibtex
+@article{xiao2026joyai,
+  title={JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion},
+  author={Xiao, Yicheng and Dai, Wenxun and Qin, Xinran and Song, Lin and Zhang, Maoquan and Xu, Hang and Chen, Yukang and Li, Yitong and Zhang, Guohui and Zhang, Yuan and Zhang, Xuying and Zhang, Tommy and Yuan, Jianlong and Li, Peihao and Lu, Shuai and Fu, Siming and Zhao, Chuyang and Han, Xin and Huang, Jie and Li, Wenbo and Ma, Guoqing and Huang, Wei and Qi, Xiaojuan and Huang, Haoyang and Duan, Nan},
+  journal={arXiv preprint arXiv:2608.03974},
+  year={2026}
+}
+```
+
+---
+
+## 📄 License
 
+JoyAI-Video-Edit is released under the **Apache License 2.0**.
 
-## License Agreement
+Please refer to the project repository for the complete license:
 
-JoyAI-Video-Edit is licensed under Apache 2.0.
+https://github.com/jd-opensource/JoyAI-Video-Edit/blob/main/LICENSE