Instructions to use ControlLLM/Llama-3.1-8B-SynE-FPT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ControlLLM/Llama-3.1-8B-SynE-FPT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ControlLLM/Llama-3.1-8B-SynE-FPT")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ControlLLM/Llama-3.1-8B-SynE-FPT", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ControlLLM/Llama-3.1-8B-SynE-FPT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ControlLLM/Llama-3.1-8B-SynE-FPT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Llama-3.1-8B-SynE-FPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ControlLLM/Llama-3.1-8B-SynE-FPT

SGLang

How to use ControlLLM/Llama-3.1-8B-SynE-FPT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ControlLLM/Llama-3.1-8B-SynE-FPT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Llama-3.1-8B-SynE-FPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ControlLLM/Llama-3.1-8B-SynE-FPT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ControlLLM/Llama-3.1-8B-SynE-FPT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ControlLLM/Llama-3.1-8B-SynE-FPT with Docker Model Runner:
```
docker model run hf.co/ControlLLM/Llama-3.1-8B-SynE-FPT
```

Control-LLM-Llama3.1-8B-SynE-Full-Parameter-Tuning

This is a fine-tuned model of Llama-3.1-8B for muliligual-Chinese tasks on SynE dataset.

Linked Paper

This model is associated with the paper: Control LLM: Controlled Evolution for Intelligence Retention in LLM.

Linked Open Source code - training, eval and benchmark

This model is associated with the github: Control-LLM.

Evaluation Results

Here is an overview of the evaluation results and findings:

Benchmark Results Table

The table below summarizes evaluation results across Chinese tasks and original capabilities.

Model	CEval	CEvalC	CMMLU	CMMLUC	C-Avg	BBH	MLU	MLUP	O-Avg	Overall
Llama3.1-8B	48.3	12.8	51.1	14.1	13.9	65.2	65.4	35.5	45.9	29.9
Llama-3-SynE	57.7	22.3	57.1	22.8	22.8	61.9	64.0	32.6	42.9	32.9
Full Param Tune	59.0	40.2	60.2	44.3	43.8	64.8	64.9	35.0	45.4	44.6
Stack Expansion	56.0	32.7	55.2	33.4	33.3	62.3	65.6	35.3	44.8	39.1
Concat-Lerp*	57.1	34.8	57.0	37.4	37.1	64.4	64.6	35.8	45.9	41.5
Hybrid Expansion	58.9	44.7	57.9	44.3	44.4	65.1	65.7	36.9	46.8	45.6
Control LLM*	57.0	44.7	56.0	44.9	44.8	68.2	65.6	37.9	48.5	46.7

Explanation:

CEval: Chinese Evaluation
CEvalC: Chinese Evaluation (CoT - Chain of Thought)
CMMLU: Chinese MMLU
CMMLUC: Chinese MMLU (CoT)
C-Avg: Chinese - Size Weighted Average across CEval, CEvalC, CMMLU, and CMMLUC
BBH: BigBench Hard
MLU: MMLU (Massive Multitask Language Understanding)
MLUP: MMLU Pro
O-Avg: Original Capability - Size Weighted Average across BBH, MLU, and MLUP
Overall: Combined average across all tasks

Full Parameter Tuning on Chinese-SynE

The following plot illustrates the Catastrophic Forgetting of full parameter tuning in terms of hidden states alignment drift.

Downloads last month: 14

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for ControlLLM/Llama-3.1-8B-SynE-FPT

Base model

meta-llama/Llama-3.1-8B

Finetuned

(1786)

this model

Datasets used to train ControlLLM/Llama-3.1-8B-SynE-FPT

Paper for ControlLLM/Llama-3.1-8B-SynE-FPT

Control LLM: Controlled Evolution for Intelligence Retention in LLM

Paper • 2501.10979 • Published Jan 19, 2025 • 6

Evaluation results

exact_match,strict-match (meta_pretrain) on Pretraining Evaluation Dataset
self-reported

0.454
exact_match,strict-match (meta_bbh_3shot_cot_pretrain) on Pretraining Evaluation Dataset
self-reported

0.648
acc,none (meta_mmlu_5shot_pretrain) on Pretraining Evaluation Dataset
self-reported

0.649
exact_match,strict-match (meta_mmlu_pro_5shot_pretrain) on Pretraining Evaluation Dataset
self-reported

0.350
acc,none (ceval-valid) on Chinese Evaluation Dataset
self-reported

0.590
exact_match,strict-match (ceval-valid-pretrain-cot_zh) on Chinese Evaluation Dataset
self-reported

0.402
acc,none (cmmlu) on Chinese Evaluation Dataset
self-reported

0.602
exact_match,strict-match (cmmlu_pretrain_cot_zh) on Chinese Evaluation Dataset
self-reported

0.443