2-bit copy in VRAM + exact residual in system RAM as the verifier: speculative decoding for models that don't fit your GPU

1. 

   消息推送

   2-bit copy in VRAM + exact residual in system RAM as the verifier: speculative decoding for models that don't fit your GPU

   22小时前

   2-bit copy in VRAM + exact residual in system RAM as the verifier: speculative decoding for models that don't fit your GPU
   
   🌐 链接: https://www.reddit.com/r/LocalLLaMA/comments/1xc55z3/2bit_copy_in_vram_exact_residual_in_system_ram_as/
   🏷️ 分组: Reddit｜AI·自托管·VPS
   🕒 时间: 2026-10-22 09:18:28

   Reddit

   

   From the LocalLLaMA community on Reddit: 2-bit copy in VRAM + exact residual in system RAM as the verifier: speculative decoding…

   Explore this post and more from the LocalLLaMA community

   22小时前