speculative-prefill

Here are 2 public repositories matching this topic...

Luce-Org / lucebox

LLM speculative inference server for consumer & heterogeneous hardware

spark kernel cuda cuda-kernels rocm heterogeneous-computing luce poolside rtx3090 llama-cpp local-ai qwen speculative-decoding strix-halo dflash megakernel speculative-prefill pflash r9700

Updated Aug 22, 2026
C++

JohnScheuer / speculative-prefill-bench

Star

Event-driven benchmark of speculative prefill during user reading time in multi-turn LLM conversations, measuring net TTFT benefit, contention penalty, and the conditions under which speculation becomes net-negative.

python performance-engineering benchmark simulation latency scheduler inference systems gating serving multi-turn prefill event-driven-simulation contention kv-cache ttft llm speculative-prefill

Updated Jul 24, 2026
Python

Improve this page

Add a description, image, and links to the speculative-prefill topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the speculative-prefill topic, visit your repo's landing page and select "manage topics."

Learn more