Instructions to use StarDoc-AI/NaviDC-OCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use StarDoc-AI/NaviDC-OCR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="StarDoc-AI/NaviDC-OCR", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("StarDoc-AI/NaviDC-OCR", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("StarDoc-AI/NaviDC-OCR", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use StarDoc-AI/NaviDC-OCR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "StarDoc-AI/NaviDC-OCR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StarDoc-AI/NaviDC-OCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/StarDoc-AI/NaviDC-OCR

SGLang

How to use StarDoc-AI/NaviDC-OCR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "StarDoc-AI/NaviDC-OCR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StarDoc-AI/NaviDC-OCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "StarDoc-AI/NaviDC-OCR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "StarDoc-AI/NaviDC-OCR",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use StarDoc-AI/NaviDC-OCR with Docker Model Runner:
```
docker model run hf.co/StarDoc-AI/NaviDC-OCR
```

Add Real5-OmniDocBench evaluation results for NaviDC-OCR

by changdazhou - opened 7 days ago

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+48

-0

changdazhou

7 days ago

Real5-OmniDocBench evaluation results for NaviDC-OCR

This PR adds the evaluation file for NaviDC-OCR under the Hugging Face
evaluation-results workflow.

This is based on the new evaluation results feature: https://huggingface.co/docs/hub/eval-results.

Dataset

PaddlePaddle/Real5-OmniDocBench

File

.eval_results/real5_omnidocbench.yaml

Tasks

overall
scanning
warping
screen_photography
illumination
skew

Add Real5-OmniDocBench evaluation results for NaviDC-OCRbf260fd1

caipeng328

StarDoc-AI org 7 days ago

Thank you for your interest in our work. However, there is a significant discrepancy between your evaluation results and our own evaluation results. If you are willing to share your evaluation results with us, we would be more than happy to analyze the issue. The table below shows our evaluation results.

caipeng328

StarDoc-AI org 7 days ago

Thank you for your interest. This is the Markdown file from our local evaluation. I’m curious to understand where the discrepancy comes from. If you are willing to provide some detailed evaluation results, that would be very helpful for analyzing the discrepancy.
https://drive.google.com/file/d/1XihsGFTfmhqTzogMJkV07K7XINNsbAQO/view

changdazhou

6 days ago

Thank you very much for your reply. Based on the results you provided, we conducted a repeat test and verification, and the results are largely consistent with the metrics reported in our PR. Therefore, the issue may lie in your evaluation environment.

Real5-OmniDocBench is strictly aligned with OmniDocBench V1.5. Therefore, both the evaluation environment and evaluation code should strictly follow the OmniDocBench V1.5 version for evaluation.

caipeng328 changed pull request status to closed 6 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment