README.md

metadata

title: MLSE Player 3D Generator
emoji: 🏀
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false

MLSE Player 3D Generator

A 3D player model generator that uses Meta's SAM 3D Body model to convert images of athletes into detailed 3D models.

Overview

This application allows you to upload images of players and automatically generate 3D human body models. It uses Meta's SAM 3D Body technology, which is designed for high-quality 3D human mesh reconstruction from a single image.

Features

• Upload player images and generate 3D models
• Automatic player detection and segmentation
• High-quality 3D human mesh reconstruction
• Interactive 3D viewer for examining models
• Export models in GLB format for use in various applications

Setup

Prerequisites

• Access to Hugging Face model repository
• GitHub account for Hugging Face Spaces

Deployment Options

1. Hugging Face Spaces (Recommended)

1. Fork this repository on GitHub
2. Create a new Space on Hugging Face Spaces (https://huggingface.co/spaces)
3. Link your GitHub repository to the Space
4. Choose "Docker" as the Space SDK
5. Configure the Space with GPU hardware (required for optimal performance)
6. Set up the Hugging Face access token as a secret named HF_TOKEN
7. Deploy the Space

2. Local Development

1. Clone this repository
2. Install dependencies:

   pip install -r requirements.txt
   
   # Install Detectron2
   pip install 'git+https://github.com/facebookresearch/detectron2.git@a1ce2f9' --no-build-isolation --no-deps
   
   # Optional: Install MoGe
   pip install git+https://github.com/microsoft/MoGe.git
   

3. Run the application:

   python -m app.main
   

Environment Variables

• HF_TOKEN: Your Hugging Face access token (required for model download)
• PORT: The port to run the server on (default: 7860)

Usage

API Endpoints

Upload an Image

POST /api/upload

Form data:

• file: Image file (JPEG/PNG)
• player_name: Name for the model (default: "player")
• use_keypoints: Whether to use keypoint detection (default: true)
• use_mask: Whether to use segmentation masks (default: true)

Process Base64 Image

POST /api/process

JSON body:

{
  "image_data": "base64_encoded_image_data",
  "player_name": "player_name",
  "options": {
    "use_keypoints": true,
    "use_mask": true
  }
}

Check Job Status

POST /api/status

JSON body:

{
  "job_id": "job_id_from_upload_response"
}

List All Jobs

GET /api/jobs

Get Model File

GET /api/model/{job_id}

Integration with MLSE Platform

To integrate this service with the MLSE platform:

1. Deploy this service on Hugging Face Spaces
2. Add the Player 3D Generator to the MLSE AI tools section
3. Configure the frontend to call this service's API endpoints
4. Use the returned model URLs to display 3D models in the MLSE platform

Technical Details

Model Information

This application uses SAM 3D Body, a foundation model for 3D human body reconstruction from Meta AI:

• Model: facebook/sam-3d-body-dinov3 (or facebook/sam-3d-body-vith)
• Papers: SAM 3D: Segment Anything in 3D with Momentum Human Rig

License

This project uses SAM 3D Body which is covered by Meta's license. Please check the original repository for license details.