#distributed-inference (6 Repositories)
Ranked open-source repositories tagged with #distributed-inference, scored by pull request acceptance likelihood and maintainer engagement velocity.
67.8%
5.5h
6 repositories tagged #distributed-inference
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
llm-d/llm-d
Achieve state of the art inference performance with modern accelerators on Kubernetes
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
awslabs/awsome-distributed-ai
Best practices, reference architectures, and examples for distributed AI training and inference on AWS.
Nehanth/pooled
Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
Nehanth/swarmllm
Every device brings a slice. Together they run the whole model. Peer-to-peer LLM inference across browser tabs: a from-scratch WebGPU engine and a WebRTC runtime that split a 27B model over the devices in a room.