Motion VQ-VAE (regular) model update
Browse files
README.md
CHANGED
|
@@ -34,7 +34,7 @@ MotionVQVAE learns to compress human motion sequences into discrete tokens using
|
|
| 34 |
### Multi-Group VQ Architecture
|
| 35 |
|
| 36 |
Instead of a single codebook, MG-VQVAE uses **64 parallel groups**, each with its own 512-code codebook. This provides:
|
| 37 |
-
- Effective codebook size: 512^64
|
| 38 |
- Fine-grained control over different motion aspects
|
| 39 |
- Better reconstruction quality through distributed quantization
|
| 40 |
|
|
@@ -185,6 +185,43 @@ Final evaluation metrics at 300K iterations:
|
|
| 185 |
| Joint Velocity | 0.02301 |
|
| 186 |
| Joint Rotation | 0.00383 |
|
| 187 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 188 |
## Files in This Repository
|
| 189 |
|
| 190 |
| File | Size | Description |
|
|
|
|
| 34 |
### Multi-Group VQ Architecture
|
| 35 |
|
| 36 |
Instead of a single codebook, MG-VQVAE uses **64 parallel groups**, each with its own 512-code codebook. This provides:
|
| 37 |
+
- Effective codebook size: $512^{64} \approx 2.47 \times 10^{173}$ combinations
|
| 38 |
- Fine-grained control over different motion aspects
|
| 39 |
- Better reconstruction quality through distributed quantization
|
| 40 |
|
|
|
|
| 185 |
| Joint Velocity | 0.02301 |
|
| 186 |
| Joint Rotation | 0.00383 |
|
| 187 |
|
| 188 |
+
## Evaluation Results
|
| 189 |
+
|
| 190 |
+
Quantitative evaluation on the MotionMillion test set using rotation-based reconstruction (global positions computed via forward kinematics).
|
| 191 |
+
|
| 192 |
+
### Rotation & Position Errors
|
| 193 |
+
|
| 194 |
+
| Metric | Value |
|
| 195 |
+
|--------|-------|
|
| 196 |
+
| **Root Position RMSE** | 0.0191 m (19.1 mm) |
|
| 197 |
+
| **Root Rotation Error** | 1.36° |
|
| 198 |
+
| **Joint Rotation Error (avg)** | 1.34° |
|
| 199 |
+
| **MPJPE** | 77.0 mm |
|
| 200 |
+
|
| 201 |
+
### Motion Quality
|
| 202 |
+
|
| 203 |
+
| Metric | Value | Rating |
|
| 204 |
+
|--------|-------|--------|
|
| 205 |
+
| **Jitter Ratio** (Pred/GT) | 7.073 | ⚠️ Very High Jitter |
|
| 206 |
+
|
| 207 |
+
### Feature-Space MSE (Denormalized)
|
| 208 |
+
|
| 209 |
+
| Feature Group | MSE |
|
| 210 |
+
|---------------|-----|
|
| 211 |
+
| Root XZ Velocity | 0.000078 |
|
| 212 |
+
| Root Rotation (6D) | 0.000214 |
|
| 213 |
+
| Joint Positions | 0.003319 |
|
| 214 |
+
| Joint Velocities | 0.000386 |
|
| 215 |
+
| Joint Rotations (6D) | 0.000465 |
|
| 216 |
+
| **Total** | **0.001130** |
|
| 217 |
+
|
| 218 |
+
### Codebook Utilization
|
| 219 |
+
|
| 220 |
+
| Metric | Value |
|
| 221 |
+
|--------|-------|
|
| 222 |
+
| **Perplexity** | 265.3 |
|
| 223 |
+
| **Codebook Usage** | 79.81% |
|
| 224 |
+
|
| 225 |
## Files in This Repository
|
| 226 |
|
| 227 |
| File | Size | Description |
|
mean.npy
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1216
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8f17f33d27b65ba4bef3f87889df0ab22bfaabc14a161cfcc9dc06ab6ba0819e
|
| 3 |
size 1216
|
pytorch_model.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 293120435
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:748e60e09ec022ce08e5e336194bf9d3280076502979a6486dddd063c0f502d5
|
| 3 |
size 293120435
|
std.npy
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1216
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ebc010a5c08c2add265de1295490eee4b5b796534c38eb17cd7ca464c6538999
|
| 3 |
size 1216
|