Instructions to use bfpill/gemma-3-270m-sft-skywork-chosen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bfpill/gemma-3-270m-sft-skywork-chosen with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bfpill/gemma-3-270m-sft-skywork-chosen")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bfpill/gemma-3-270m-sft-skywork-chosen")
model = AutoModelForCausalLM.from_pretrained("bfpill/gemma-3-270m-sft-skywork-chosen", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bfpill/gemma-3-270m-sft-skywork-chosen with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bfpill/gemma-3-270m-sft-skywork-chosen"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bfpill/gemma-3-270m-sft-skywork-chosen",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/bfpill/gemma-3-270m-sft-skywork-chosen

SGLang

How to use bfpill/gemma-3-270m-sft-skywork-chosen with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bfpill/gemma-3-270m-sft-skywork-chosen" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bfpill/gemma-3-270m-sft-skywork-chosen",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bfpill/gemma-3-270m-sft-skywork-chosen" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bfpill/gemma-3-270m-sft-skywork-chosen",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use bfpill/gemma-3-270m-sft-skywork-chosen with Docker Model Runner:
```
docker model run hf.co/bfpill/gemma-3-270m-sft-skywork-chosen
```

gemma-3-270m SFT'd on Skywork-Reward-Preference-80K (chosen-only)

A response-only SFT of google/gemma-3-270m on the chosen responses of Skywork/Skywork-Reward-Preference-80K-v0.2.

This is the anchor checkpoint used as the SFT reference policy for a research project on data-space DPO — applying preference signal by reweighting SFT samples through a susceptibility matrix, instead of via DPO's weight-space gradient descent. It is not intended as a useful chat model — at 270M parameters and a single epoch of SFT, generations are at best "coherent text on the topic of the prompt". The role of this checkpoint is to be a controlled, reproducible anchor for sampling-based experiments.

Training recipe


Base	`google/gemma-3-270m` (pretrained, no chat template)
Data	`Skywork/Skywork-Reward-Preference-80K-v0.2`, chosen responses only (~77K examples)
Epochs	1
Learning rate	2e-5, cosine schedule, 3% warmup
Batch size	16 per device × 4 GPUs = 64 effective
Precision	bf16 + gradient checkpointing
Loss	response-only cross-entropy (prompt tokens masked to -100)
Total steps	602
Final train loss	1.41
Wall time	~4 min on 8× H100

Prompt format

The base model has no chat_template, so we use a fixed plain-text format matching the SFT distribution:

Human: <user message>
Assistant: ASSISTANT_RESPONSE

Internally the prompt prefix is "Human: <user>\n\nAssistant:" and the loss is computed only on the tokens following Assistant:.

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "bfpill/gemma-3-270m-sft-skywork-chosen"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype=torch.bfloat16).cuda().eval()

prompt = f"Human: What is the capital of France?\n\nAssistant:"
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(
    **ids, max_new_tokens=80, do_sample=False,
    repetition_penalty=1.15, pad_token_id=tok.eos_token_id,
)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))

Limitations

270M params, 1 epoch of SFT on ~77K examples. Generations are short and often factually incorrect.
Trained exclusively on the Skywork prompt distribution; performance on other distributions (instruction-following, dialogue, code, math) reflects the base model plus a thin Skywork-flavoured veneer.
Not preference-tuned. The whole point of this checkpoint is to be the anchor for a follow-up data-space patterning experiment that is the replacement for the usual DPO step.
Inherits the base model's biases and license restrictions.

License

Gemma Terms of Use — see Google's Gemma license. This SFT preserves the base model's weights structure; usage is subject to the same terms.

Source code

The trainer that produced this checkpoint: rm-patterning/policy/sft_anchor.py in timaeus-research/timaeus@max/rm-scaling (commit 3a3e0755e and later).

torchrun --nproc_per_node=8 rm-patterning/policy/sft_anchor.py \
    --base-model google/gemma-3-270m \
    --output-dir ./gemma-3-270m-sft-skywork-chosen \
    --epochs 1 --learning-rate 2e-5 --batch-size 16 --max-length 1024

Downloads last month: 17

Safetensors

Model size

0.3B params

Tensor type

BF16

Model tree for bfpill/gemma-3-270m-sft-skywork-chosen

Base model

google/gemma-3-270m

Finetuned

(161)

this model

bfpill
/

gemma-3-270m-sft-skywork-chosen