Instructions to use abhinand/gemma-2b-it-tamil-v0.1-alpha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use abhinand/gemma-2b-it-tamil-v0.1-alpha with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="abhinand/gemma-2b-it-tamil-v0.1-alpha")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("abhinand/gemma-2b-it-tamil-v0.1-alpha")
model = AutoModelForCausalLM.from_pretrained("abhinand/gemma-2b-it-tamil-v0.1-alpha", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use abhinand/gemma-2b-it-tamil-v0.1-alpha with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "abhinand/gemma-2b-it-tamil-v0.1-alpha"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhinand/gemma-2b-it-tamil-v0.1-alpha",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/abhinand/gemma-2b-it-tamil-v0.1-alpha

SGLang

How to use abhinand/gemma-2b-it-tamil-v0.1-alpha with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "abhinand/gemma-2b-it-tamil-v0.1-alpha" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhinand/gemma-2b-it-tamil-v0.1-alpha",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "abhinand/gemma-2b-it-tamil-v0.1-alpha" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "abhinand/gemma-2b-it-tamil-v0.1-alpha",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use abhinand/gemma-2b-it-tamil-v0.1-alpha with Docker Model Runner:
```
docker model run hf.co/abhinand/gemma-2b-it-tamil-v0.1-alpha
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Gemma 2B Tamil v0.1 Alpha [Experimental Release]

This is a Tamil instruction finetuned version of Google's Gemma 2B model. This is an experiment to see if Gemma can be adapted for Tamil without expanding vocabulary. While the responses may be rusty at times, it shows a lot of promise for a 2B parameter model.

Procedure:

The Gemma base model was continually pretrained on all available Tamil Wikipedia data for 3 epochs.
The updated model was then finetuned on a mix of English and Tamil alpaca datasets for 5 epochs.

Note: This project is currently under development (FOR TAMIL). The initial pretraining phase may not have been extensive enough, which suggests that the model's performance could improve by extending the pretraining on a larger dataset, such as CulturaX.

🏆 Benchmarks

This model outperforms Google's Gemma 2B base and instruct models on all benchmarks in Nous evaluation suite. It also surprisingly outperforms mlabonne/Gemmalpaca-2B (the best performing 2B model in benchmarks as of Feb 25, 2024) despite being a model aimed at language adaptation.

Model	Average	AGIEval	GPT4All	TruthfulQA	Bigbench
gemma-2b-it-tamil-v0.1-alpha 📄	39.41	23.38	58.94	43.18	32.14
mlabonne/Gemmalpaca-2B 📄	38.39	24.48	51.22	47.02	30.85
google/gemma-2b-it 📄	36.1	23.76	43.6	47.64	29.41
google/gemma-2b 📄	34.26	22.7	43.35	39.96	31.03

Model description

Model type: A 2B parameter GPT-like model finetuned on 100,000 samples consisting of an equal proportion of English and Tamil samples.
Language(s): Bilingual. English and Tamil.
License: Google Gemma Terms of Use
Finetuned from model: abhinand/gemma-2b-tamil
Training Precision: bfloat16
Training Hardware: 4x Nvidia RTX 3090 GPUs
Training Cost: $20

Support my work

If you appreciate this work and would like to support its continued development, consider buying me a coffee. Your support is invaluable and greatly appreciated.

Prompting Format [Alpaca]

Prompt Template Without Input

{system_prompt}

### Instruction:
{instruction or query}

### Response:
{response}

Prompt Template With Input

{system_prompt}

### Instruction:
{instruction or query}

### Input:
{input}

### Response:
{response}

Usage Note

It's important to note that the models have not undergone detoxification. Therefore, while they possess impressive linguistic capabilities, there is a possibility for them to generate content that could be deemed harmful or offensive. We urge users to exercise discretion and supervise the model's outputs closely, especially in public or sensitive applications.

Meet the Developers

Get to know the creators behind this innovative model and follow their contributions to the field:

Abhinand Balachandran

We hope this model serves as a valuable tool in your NLP toolkit and look forward to seeing the advancements it will enable in the understanding and generation of the Tamil language.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	47.60
AI2 Reasoning Challenge (25-Shot)	50.09
HellaSwag (10-Shot)	71.41
MMLU (5-Shot)	39.94
TruthfulQA (0-shot)	42.63
Winogrande (5-shot)	64.96
GSM8k (5-shot)	16.60

Downloads last month: 359

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for abhinand/gemma-2b-it-tamil-v0.1-alpha

Base model

google/gemma-2b

Finetuned

abhinand/gemma-2b-tamil

Finetuned

(2)

this model

Quantizations

2 models

Datasets used to train abhinand/gemma-2b-it-tamil-v0.1-alpha

Spaces using abhinand/gemma-2b-it-tamil-v0.1-alpha 5

Evaluation results

normalized accuracy on AI2 Reasoning Challenge (25-Shot)
test set Open LLM Leaderboard

50.090
normalized accuracy on HellaSwag (10-Shot)
validation set Open LLM Leaderboard

71.410
accuracy on MMLU (5-Shot)
test set Open LLM Leaderboard

39.940
mc2 on TruthfulQA (0-shot)
validation set Open LLM Leaderboard

42.630
accuracy on Winogrande (5-shot)
validation set Open LLM Leaderboard

64.960
accuracy on GSM8k (5-shot)
test set Open LLM Leaderboard

16.600