Instructions to use bharatgenai/patram-7b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bharatgenai/patram-7b-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bharatgenai/patram-7b-instruct", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("bharatgenai/patram-7b-instruct", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use bharatgenai/patram-7b-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bharatgenai/patram-7b-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bharatgenai/patram-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bharatgenai/patram-7b-instruct

SGLang

How to use bharatgenai/patram-7b-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bharatgenai/patram-7b-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bharatgenai/patram-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bharatgenai/patram-7b-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bharatgenai/patram-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bharatgenai/patram-7b-instruct with Docker Model Runner:
```
docker model run hf.co/bharatgenai/patram-7b-instruct
```

Unable to determine the version compatibilities

by AlfredManan - opened Mar 30

Discussion

AlfredManan

Mar 30

Hi BharatGen Team,

I am trying to run inference using patram-7b-instruct locally, but I am running into multiple errors due to library version incompatibilities. The model card does not specify exact pinned versions for the required libraries, which makes it very difficult to set up a working environment.

Errors encountered so far:

ImportError: cannot import name 'TextKwargs' from 'transformers.processing_utils'
→ Caused by older versions of transformers that do not have TextKwargs
AttributeError: 'NoneType' object has no attribute 'size' at modeling_patram.py:2320
→ past_key_values is None on the first forward pass, caused by newer transformers versions (≥ 4.47) that changed how KV cache is initialized
AssertionError at modeling_patram.py:2669 — assert generation_config.use_cache
→ The model hardcodes use_cache=True, but newer transformers versions conflict with this

My setup:

GPU: 22 GB VRAM
Python: 3.10
OS: Linux

Request:
Could you please share a complete requirements.txt with exact pinned versions for all dependencies that are confirmed to work with this model? Specifically:

transformers==?
torch==?
accelerate==?
pydantic==?
Any other pinned versions

This will help the community avoid spending hours debugging environment issues and focus on actually using the model.

Thank you!

hrithiksagar-bgen

BharatGen AI org Mar 31

•

edited Apr 7

Hi, thanks for sharing the detailed error trace and environment details.

For patram-7b-instruct, the dependency versions we currently recommend for local inference are:

torch==2.7.0
torchvision==0.22.0
transformers==4.50.3
accelerate==0.26.0
einops==0.8.1

The errors you ran into are generally due to version mismatches across the transformers stack, especially around cache handling and generation behavior across releases.

Using a clean environment with the pinned versions above should resolve these issues.

We’ve also added a requirements.txt file to the repository files section so the recommended setup is easier to reproduce.

Thanks for reporting this.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment