Daniel (Unsloth)'s picture

Daniel (Unsloth) PRO

danielhanchen

·

https://unsloth.ai/

AI & ML interests

None yet

Recent Activity

replied to their post about 10 hours ago

You can now run Kimi K2.5 locally! 🔥 We shrank the 1T model to 240GB (-60%) via Dynamic 1-bit. Get >40 tok/s on 242GB or 622GB VRAM/RAM for near full precision. GGUF: https://huggingface.co/unsloth/Kimi-K2.5-GGUF Guide: https://unsloth.ai/docs/models/kimi-k2.5

replied to their post about 10 hours ago

You can now run Kimi K2.5 locally! 🔥 We shrank the 1T model to 240GB (-60%) via Dynamic 1-bit. Get >40 tok/s on 242GB or 622GB VRAM/RAM for near full precision. GGUF: https://huggingface.co/unsloth/Kimi-K2.5-GGUF Guide: https://unsloth.ai/docs/models/kimi-k2.5

liked a model about 10 hours ago

unsloth/DeepSeek-V3.2-GGUF

View all activity

Organizations

New activity in unsloth/DeepSeek-V3.2-GGUF about 10 hours ago

Does this support DSA/lighting attention?

#1 opened 1 day ago by

Already testing it

#2 opened about 12 hours ago by

New activity in unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF about 14 hours ago

New Refresh with added Tool Calling in calibration dataset and improved imatrix

#21 opened 1 day ago by

New activity in unsloth/Kimi-K2.5-GGUF about 19 hours ago

Q4_K_S versus Q3_K_XL

#3 opened 1 day ago by

New activity in unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF 1 day ago

Performance is poor

#1 opened 6 days ago by

New activity in unsloth/DeepSeek-OCR-2 1 day ago

Unsloth implementation

#1 opened 4 days ago by

New activity in unsloth/Trinity-Large-Preview-GGUF 1 day ago

Quantasation for Ming-flash-omni-Preview?

#1 opened 2 days ago by

facehuggerfromspace

New activity in unsloth/Kimi-K2.5-GGUF 2 days ago

Original model is just 600gb

#2 opened 2 days ago by

New activity in unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF 2 days ago

The generation falls into constant repetition without any good result

#2 opened 6 days ago by

New activity in unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF 3 days ago

Fix typos in README.md

#3 opened 6 days ago by

New activity in unsloth/GLM-4.7-Flash-FP8-Dynamic 3 days ago

vllm nightly currently not supporting Blackwell with this model

#3 opened 3 days ago by

dual 3090 inference

#1 opened 4 days ago by

New activity in unsloth/GLM-4.7-Flash-GGUF 3 days ago

New flag: kv-unified. Performance report: 90 t/s on RTX 4090D 48GB

#18 opened 3 days ago by

New activity in unsloth/GLM-4.7-Flash-FP8-Dynamic 3 days ago

Severe Looping/Repetitive Output when using --kv-cache-dtype fp8 with GLM-4.7-Flash-FP8-Dynamic on vLLM

#2 opened 4 days ago by

New activity in deepseek-ai/DeepSeek-OCR-2 3 days ago

Guide to Run + Fine-tune DeepSeek-OCR 2

#4 opened 4 days ago by

New activity in unsloth/GLM-4.7-Flash-GGUF 5 days ago

Fix GLM 4.7 Lite MoE gating func

#9 opened 10 days ago by

New activity in unsloth/ERNIE-4.5-21B-A3B-PT-GGUF 6 days ago

update the gguf

#4 opened 3 months ago by

New activity in unsloth/GLM-4.7-Flash-GGUF 7 days ago

DISABLE REPEAT PENALTY or set = 1.0

#13 opened 7 days ago by

new ggufs working as intended

#12 opened 7 days ago by

New activity in unsloth/LTX-2-GGUF 8 days ago

New Sample Workflow!!

#7 opened 10 days ago by