I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)

🌐 链接: https://www.reddit.com/r/LocalLLaMA/comments/1xhog7b/i_trained_a_21b_moe_336b_active_from_scratch_on/
🏷️ 分组: Reddit｜AI·自托管·VPS
🕒 时间: 2026-10-28 21:43:43
Reddit
From the LocalLLaMA community on Reddit: I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)
From the LocalLLaMA community on Reddit: I trained a 21B MoE (3.36B active) from scratch on 70B tokens (emergent ability observations)
Explore this post and more from the LocalLLaMA community