#kv-cache (8 Repositories)
Ranked open-source repositories tagged with #kv-cache, scored by pull request acceptance likelihood and maintainer engagement velocity.
44.6%
17.0h
8 repositories tagged #kv-cache
feichai0017/loom-infer
Rust-native GPU operator library for LLM inference, built with cuda-oxide
novitalabs/pegaflow
High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.
syv-ai/qwen38-27b-rtx3090
Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks
alibaba/tair-kvcache
Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
huawei-csl/KVarN
KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
feichai0017/orbitkv
OrbitKV: a Rust attention-state compiler and lifetime-safe KV block manager.
quantumaikr/quant.cpp
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.