view article Article An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct leonardlin • Jun 11, 2024 • 69
view post Post 1374 📈 One of the biggest changes in Llama 3 was the training dataset, which grew by 7X over Llama 2 (2T to 15T tokens) 🚀While Meta did not open source the dataset, it sparked a thought... what would happen if everyone had access to a big, high-quality dataset? 🤔To address that, in April this year, @huggingface released FineWeb, a 15T token open-source dataset 🌍And now they are releasing FineWeb Technical Report and FineWeb Edu 📚🏆 15T tokens in FineWeb outperforming other open datasets🎓 1.3T highest-quality educational dataset FineWeb-Edu📘 5.4T high-quality educational tokens in FineWeb-Edu-2FineWeb Edu outperforms other datasets on MMLU, ARC, OpenBookQA 📈ODC-By 1.0 license 📜Report: HuggingFaceFW/blogpost-fineweb-v1 🚀 3 3 + Reply