Moving a running model's layers between GPUs mid-reply (8 ms, no re-prefill)

🌐 链接: https://www.reddit.com/r/LocalLLaMA/comments/1x5i7o9/moving_a_running_models_layers_between_gpus/
🏷️ 分组: Reddit｜AI·自托管·VPS
🕒 时间: 2026-10-14 12:40:53
Reddit
From the LocalLLaMA community on Reddit: Moving a running model's layers between GPUs mid-reply (8 ms, no re-prefill)
From the LocalLLaMA community on Reddit: Moving a running model's layers between GPUs mid-reply (8 ms, no re-prefill)
Explore this post and more from the LocalLLaMA community