Instructions to use slovak-nlp/mistral-sk-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use slovak-nlp/mistral-sk-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="slovak-nlp/mistral-sk-7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("slovak-nlp/mistral-sk-7b")
model = AutoModelForCausalLM.from_pretrained("slovak-nlp/mistral-sk-7b")

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use slovak-nlp/mistral-sk-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "slovak-nlp/mistral-sk-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slovak-nlp/mistral-sk-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/slovak-nlp/mistral-sk-7b

SGLang

How to use slovak-nlp/mistral-sk-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "slovak-nlp/mistral-sk-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slovak-nlp/mistral-sk-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "slovak-nlp/mistral-sk-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "slovak-nlp/mistral-sk-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use slovak-nlp/mistral-sk-7b with Docker Model Runner:
```
docker model run hf.co/slovak-nlp/mistral-sk-7b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for mistral-sk-7b

mistral-sk-7b is a Slovak language version of the Mistral-7B-v0.1 large language model with 7 billion parameters.

Model Details

mistral-sk-7b is a Slovak language model obtained by full parameter finetuning of the Mistral-7B-v0.1 large language model with the data from the Araneum Slovacum VII Maximum web corpus. The model was developed in collaboration of Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Košice; Centre of Social and Psychological Sciences of the Slovak Academy of Sciences, Ľ. Štúr Institute of Linguistics, Slovak Academy of Sciences and ČZ, o.z.. This is a base pre-trained model that can be used for further finetuning for the downstream tasks in Slovak language. Note that this model does not have any moderation mechanisms.

Language: Slovak
License: Apache license 2.0
Finetuned from model: Mistral-7B-v0.1
Authors:
- Peter Bednár, Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Košice
- Marek Dobeš, Centre of Social and Psychological Sciences of the Slovak Academy of Sciences and ČZ o.z.
- Radovan Garabík, Ľ. Štúr Institute of Linguistics, Slovak Academy of Sciences, supported by DiusAI a. s.

Supported by

Part of the Research results was obtained using the high performance computing resources operated by CINECA and awarded within the the National Leonardo access call 2023 by the Centre of Operations, Slovak Academy of Sciences and the Slovak National Supercomputing centre.

Downloads last month: 651

Model tree for slovak-nlp/mistral-sk-7b

Base model

mistralai/Mistral-7B-v0.1

Finetuned

(1032)

this model

Adapters

2 models

Quantizations

4 models