#kv-cache (19 Repositories)
Ranked open-source repositories tagged with #kv-cache, scored by pull request acceptance likelihood and maintainer engagement velocity.
49.5%
65.2h
19 repositories tagged #kv-cache
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
alibaba/tair-kvcache
Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.
feichai0017/loom-infer
Rust-native GPU operator library for LLM inference, built with cuda-oxide
NVIDIA/kvpress
LLM KV cache compression made easy
syv-ai/qwen38-27b-rtx3090
Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks
novitalabs/pegaflow
High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.
bojieli/ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
swellweb/reame
CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
feichai0017/orbitkv
High Performance KV Cache for LLM Inference
feichai0017/oxide-infer
Rust-native GPU operator library for LLM inference, built with cuda-oxide
jjiantong/Awesome-KV-Cache-Optimization
[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
syv-ai/HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
feichai0017/loom-kernels
Rust-first high-performance CUDA operator library for LLM inference
quantumaikr/quant.cpp
LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.
thu-nics/C2C
[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"
huawei-csl/KVarN
KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
QuanjianSong/FashionChameleon
Official Pytorch Code of the Paper "FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization"
datawhalechina/zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
HDT3213/godis
A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群