OpenMHC Benchmark

OpenMHC Benchmark

OpenMHC Leaderboard

Wearable & mobile health benchmark on MyHeartCounts. Track 1 (predictive tasks) predicts weekly health outcomes — demographics, medical risk, vitals, mental well-being, and lifestyle — from 168-hour sensor embeddings. The generative tasks operate on the raw signals: Track 2a (imputation) reconstructs masked daily, minute-level signals, and Track 2b (forecasting) predicts future hourly signals. Each method is ranked by skill score vs a track baseline, computed live from the per-user evaluation substrate.

📤 Submit a model ⚙️ Code 📊 Dataset · coming soon 📄 Paper · coming soon 🏠 MyHeartCounts 🤗 Models
Cohort

Track 1 · Predictive Tasks

All participants · 1,762 users evaluated

#MethodType R ↓S ↑S_worst ↑Demo. ↑Medical ↑Vitals ↑Mental ↑Lifestyle ↑Fallback ↓Submitter
1LSM-2Wearable Foundation Models2.2+15.1+13.4+41.1-1.8+22.2+0.7+13.40.0OpenMHC team
2XGBoostStatistical Models3.5+11.6+8.3+46.9-1.4+13.6-8.5+7.30.0OpenMHC team
3MultiRocketStatistical Models3.7+7.1+4.6+27.7-4.5+5.6+0.2+6.60.0OpenMHC team
4Linear (baseline)Statistical Models4.40.00.00.00.00.00.00.00.0OpenMHC team
5WBMWearable Foundation Models4.8+3.3-0.1+22.2-2.0+6.6-7.5-2.862.8OpenMHC team
6GRU-DSupervised Neural Models4.9+3.1+0.0+18.2-6.2+10.5-3.9-3.30.0OpenMHC team
7Chronos-2Time-Series Foundation Models6.0-3.4-5.6+6.2-9.6-7.1-7.6+0.90.0OpenMHC team
8TotoTime-Series Foundation Models6.5-5.0-7.8-0.9-7.6-4.5-9.2-2.60.0OpenMHC team

Metric legend — Track 1 predicts weekly health outcomes from 168-hour sensor embeddings; scores are computed vs the Linear baseline.
R (Average Rank) — mean cross-method rank across the outcome tasks; 1 = best (lower is better).
S (Skill Score) — category-balanced % improvement over Linear across tasks (per-task AUPRC / Spearman / Pearson, paired-bootstrap mean); higher is better.
S_worst (Worst-subgroup skill) — the method's lowest skill score across the four demographic cohorts (age <40 / 40+, female / male): its performance for the worst-served subgroup; higher is better.
Demo. / Medical / Vitals / Mental / Lifestyle — per-category skill on that outcome group's tasks (Demographics; Medical Conditions & Risk; Vitals & Blood Biomarkers; Mental Well-Being; Sleep & Lifestyle), category-balanced like S.
Fallback — % of test predictions substituted by the Linear baseline when the method produced no valid output (lower is better).
Source: MyHeartCounts/OpenMHC-leaderboard-data.
Cohort — the selector above recomputes R, S, and the per-category skills within one demographic cohort (a near-median age split at 40, or sex); the count above each table is the cohort's evaluated users. S_worst is the minimum cohort skill across the four cohorts, so it reads the same in every view. Cohorts are smaller than the full population, so scores are noisier; tasks without enough users (or degenerate within the cohort) drop from the aggregates.

Submit your model

Add a method by opening a pull request on the OpenMHC leaderboard dataset that adds your per-user evaluation substrate (<track>/<method>.parquet) plus a small <method>.meta.json sidecar. Produce the substrate by running the OpenMHC eval with output_dir=…; the maintainers recompute the skill, rank, and worst-subgroup scores from it. See the step-by-step submission guide for the exact file schema.