Instructions to use DopeorNope/COKAL-DPO_test-v2-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DopeorNope/COKAL-DPO_test-v2-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DopeorNope/COKAL-DPO_test-v2-13b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DopeorNope/COKAL-DPO_test-v2-13b")
model = AutoModelForCausalLM.from_pretrained("DopeorNope/COKAL-DPO_test-v2-13b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DopeorNope/COKAL-DPO_test-v2-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DopeorNope/COKAL-DPO_test-v2-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL-DPO_test-v2-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DopeorNope/COKAL-DPO_test-v2-13b

SGLang

How to use DopeorNope/COKAL-DPO_test-v2-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DopeorNope/COKAL-DPO_test-v2-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL-DPO_test-v2-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DopeorNope/COKAL-DPO_test-v2-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DopeorNope/COKAL-DPO_test-v2-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DopeorNope/COKAL-DPO_test-v2-13b with Docker Model Runner:
```
docker model run hf.co/DopeorNope/COKAL-DPO_test-v2-13b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

(주)미디어그룹사람과숲과 (주)마커의 LLM 연구 컨소시엄으로 개발된 모델입니다

The license is cc-by-nc-sa-4.0.

🐻‍❄️COKAL-DPO_test-v2🐻‍❄️

Model Details

Model Developers Seungyoo Lee (DopeorNope)

Input Models input text only.

Output Models generate text only.

Model Architecture
COKAL-DPO_test-v2 is an auto-regressive 13B language model based on the LLaMA2 transformer architecture.

Base Model DopeorNope/COKAL_pre_DPO_Test_v1-13b

COKAL_pre_DPO_Test_v1-13b is SFT model to train DPO method

Training Dataset

DPO training dataset: DopeorNope/DPO-Ko-Dataset - private

This dataset was constructed by directly collecting and reorganizing data by DopeorNope, obtaining insights from "lvwerra/stack-exchange-paired" to create a paired dataset. (It means I do not use stack-exchange-paired; I just got an insight from it.)

SFT training dataset: DopeorNope/Orca_Near_Dedup-v2 - private

This dataset is based on "kyujinpy/OpenOrca-KO" and has been processed using the Near Dedup algorithm to remove items with a Jaccard Similarity threshold of 0.8 or higher. In addition, inconsistent inputs have been cleaned and modified.

Training
I developed the model in an environment with four RTX 3090 GPUs running Ubuntu 18.04. It seems that when uploading the model directly to a repository from a Linux server, there may be an issue causing the model to appear to have more parameters. However, this model is based on a 13B architecture.

Implementation Code


from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "DopeorNope/COKAL-DPO_test-v2"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
model_tokenizer = AutoTokenizer.from_pretrained(repo)

Acknowledgement

이 모델은 과학기술정보통신부·광주광역시가 공동 지원한 '인공지능 중심 산업융합 집적단지 조성사업'으로 지원을 받아 수행된 연구 결과입니다.
This model was supported by Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea)&Gwangju Metropolitan City.

Downloads last month: 9

Model tree for DopeorNope/COKAL-DPO_test-v2-13b

Quantizations

1 model

Dataset used to train DopeorNope/COKAL-DPO_test-v2-13b

Collection including DopeorNope/COKAL-DPO_test-v2-13b

Developed best Ko-LLM Models

Collection

These are the top models I developed on the open Ko-llm leaderboard. • 8 items • Updated Dec 24, 2023 • 2