Bitsandbytes quantization and QLORA fine-tuning

lilylii
Tue, 05 Nov 2024 21:18:44 +0000

Do you have an answer/clarification for this? I have similar confusion - I am fine-tuning Llama 3.1 with QLoRA and am unable to load the model to have tensor.dtype=torch.bfloat16

Read full topic