+- Hack Forums (https://hackforums.net)
+-- Forum: Hacks, Exploits, and Various Discussions (https://hackforums.net/forumdisplay.php?fid=45)
+--- Forum: Grayhat • Purple Team • Hybrid Security (https://hackforums.net/forumdisplay.php?fid=449)
+---- Forum: Beginner Hacking (https://hackforums.net/forumdisplay.php?fid=4)
+---- Thread: dReLU Sparsification: Recovering LLM Performance with 150B Token Pretraining (/showthread.php?tid=6320916)
dReLU Sparsification: Recovering LLM Performance with 150B Token Pretraining - Stanley - Mar 1, 2026
dReLU Sparsification: Recovering LLM Performance with 150B Token Pretraining
Learn how dReLU-based ReLUfication restores model capabilities for Mistral-7B and Mixtral-47B. Discover the high-quality pretraining datasets and mixture ratios used to achieve elite activation sparsity.
https://hackernoon.com/drelu-sparsification-recovering-llm-performance-with-150b-token-pretraining?source=rss