We built our own inference engine so a 32B model could keep 262K context inside a 16 GB GPU budget. Here's an uncut run.

🌐 链接: https://www.reddit.com/r/LocalLLaMA/comments/1xa02iw/we_built_our_own_inference_engine_so_a_32b_model/
🏷️ 分组: Reddit｜AI·自托管·VPS
🕒 时间: 2026-10-19 23:32:37
Reddit
From the LocalLLaMA community on Reddit: We built our own inference engine so a 32B model could keep 262K context inside a 16 GB…
From the LocalLLaMA community on Reddit: We built our own inference engine so a 32B model could keep 262K context inside a 16 GB…
Explore this post and more from the LocalLLaMA community