ovg-project/kvcached

This issue tracks KVCached compatibility with vLLM 0.28: adapter changes, remaining gaps, and the configurations we have actually validated. Project-wide priorities are tracked in #419; the support matrix is tracked in #425. First release priorities: v0.1.6 For the release tracked in #509, this is the first set of vLLM work I will focus on. These PRs need to land and be validated together before release; their existing individual test results are not acceptance of the combined release candidate. #492: Correct profiling order and persistent/transient memory accounting for warmup. Merged; the affected contract starts at vLLM 0.27+, not only 0.28, as Rixin clarified. This is a shared prerequisite for the 0.28/0.29 work, not a complete version-adaptation claim. #493: Packed attention KV storage and backend strides. Merged; the packed CUDA geometry applies to vLLM 0.26+, as Rixin corrected during the final review. His A100 checks covered 0.27 and 0.28, both layouts, with 48 complete nati

We need a comprehensive compatibility matrix covering the model architectures kvcached interacts with. This tracks what works, what crashes, and what produces wrong output, so users know what to expect and contributors know where to focus. Test models Architecture Model Why Standard GQA Qwen3-8B Baseline — most common setup SWA + dense hybrid GPT-OSS-20B Alternating sliding-window / full-attention layers Hybrid linear Qwen3.5-9B 3:1 GDN hybrid, some layers have no KV Cross-layer KV sharing Gemma-4-E2B-it Layers share KV with earlier layers Heterogeneous KV groups Gemma-4-12B-it Per-layer KV head count varies vLLM Model contiguous non-contiguous Qwen3-8B GPT-OSS-20B Qwen3.5-9B Gemma-4-E2B-it Gemma-4-12B-it SGLang Model contiguous non-contiguous Qwen3-8B GPT-OSS-20B Qwen3.5-9B Gemma-4-E2B-it Gemma-4-12B-it Legend: ✅ pass / ❌💥 crash at startup (link issue #) / ❌🗑️ garbled output (link issue #) / ➖

kvcached Roadmap Active workstreams, roughly in priority order. Each item links to the relevant issues/PRs; pick one up or file a new issue if something you care about isn't listed. Production stability Fix KV pool exhaustion crash under multi-instance load — #262 Central kvcached use case; current behavior is a production stability blocker. Review + merge PR #301 Serving stack compatibility [] PD (prefill / decode) disaggregation support — #302, #311 Without this, kvcached is locked out of PD-based serving stacks. vLLM exposes PD via several KVConnector implementations, each with its own assumptions about KV layout and block count; we have to validate kvcached against each pattern. NixlConnector (UCX / RDMA): PR #313 patches the two known incompatibilities (HND set_stride crash + num_blocks assertion). Pending review + merge; UCX-over-VMM transfer path still needs an end-to-end correctness run. P2pNcclConnector (NCCL P2P): untested with kvcached. No block-count asserti