Instructions to use jphme/em_german_leo_mistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jphme/em_german_leo_mistral with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jphme/em_german_leo_mistral")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jphme/em_german_leo_mistral")
model = AutoModelForCausalLM.from_pretrained("jphme/em_german_leo_mistral")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use jphme/em_german_leo_mistral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jphme/em_german_leo_mistral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_leo_mistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jphme/em_german_leo_mistral

SGLang

How to use jphme/em_german_leo_mistral with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jphme/em_german_leo_mistral" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_leo_mistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jphme/em_german_leo_mistral" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_leo_mistral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jphme/em_german_leo_mistral with Docker Model Runner:
```
docker model run hf.co/jphme/em_german_leo_mistral
```

License question

by RonanMcGovern - opened Nov 22, 2023

Discussion

RonanMcGovern

Nov 22, 2023

Cool model.

Is this trained on a public dataset?
Are synthetic datasets included?
What results in this model being only for research use?
Is this a fine-tune of the mistral instruct model OR of the base model?

jphme

Owner Nov 23, 2023

Hi Ronan,

partially (and the rest will probably be released in the future)
yes
The model is under the Apache 2.0 license of the base model, you can use it comercially at your own diescretion. I just don't recommend it and won't take any responsibility whatsoever as the model was not evaluated for productive/safe use.
base

Hope that helps :)

jphme changed discussion status to closed Nov 23, 2023

RonanMcGovern

Nov 24, 2023

Many thanks @jphme .

Two clarifications:

Was any of the data in 1) or in 2) derived from models that do not have an Apache 2 or MIT license (e.g. OpenAI and Llama have license limitations)? If this is the case, then this model would not be under Apache 2.

I know I'm getting into detail but I want to have the usage correct.

RonanMcGovern changed discussion status to open Nov 24, 2023

jphme

Owner Nov 24, 2023

Many thanks @jphme .

Two clarifications:

Was any of the data in 1) or in 2) derived from models that do not have an Apache 2 or MIT license (e.g. OpenAI and Llama have license limitations)? If this is the case, then this model would not be under Apache 2.

I know I'm getting into detail but I want to have the usage correct.

At this time, I won't go into more details on this topic or the exact data generation process, except stating that all data used for training the model was legally acquired and I, as the model creator, believe that I can rightfully publish the model under the Apache 2 license ( but without being a lawyer myself).
Please understand that this is an OpenSource project done in my spare time and I won't elaborate any more than that; even legal professionals have vastly different opinions on many specific questions in this area and I just don't have the time to discuss this.

If you need more details for commerical use, please contact me per mail - our new company Ellamind will develop customized models for business use and we will be able to train models according to your specific licensing/data limitations.

jphme changed discussion status to closed Nov 24, 2023

RonanMcGovern

Nov 24, 2023

alright, appreciate the response

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment