#llama-cpp (30 Repositories)
Ranked open-source repositories tagged with #llama-cpp, scored by pull request acceptance likelihood and maintainer engagement velocity.
72.7%
33.9h
30 repositories tagged #llama-cpp
defilantech/LLMKube
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Helldez/BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a 12 GB phone, lossless, on stock llama.cpp
mohitsoni48/TurboLLM
Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine in one command. No Electron, no Python, offline-first.
llmmanorg/llmman
Run any agent on any model, models stored as OCI images
FuJacob/cotabby
Cotabby is local AI autocomplete for your entire Mac. Open source. On device. Everywhere you type.
off-grid-ai/OGAM
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
altic-dev/FluidVoice
Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows pre-build available! A local Wispr Flow alternative. DM us on X exclusive model access! 😉 - https://x.com/fluidvoiceapp
RunanywhereAI/wally
Get up and running with GLM-5.3-flash, DeepSeek, Gemma and other open source frontier models.
StayLameBro/backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
lucasjinreal/Crane
A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..
Luce-Org/lucebox
LLM speculative inference server for heterogeneous hardware & consumer GPUs
Blackwellboy/model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
SouthpawIN/turbofit
Adaptive local inference for Hermes — TurboFit Check, evidence-built TurboFit List, repaired discovery/benchmark campaigns, Qwen 3.8 DFlash2, Bonsai DSpark, FreeToken, and Sirvir.
Osmantic/ODS
ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.
raketenkater/ggrun
llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.
mybigday/llama.rn
React Native binding of llama.cpp
SergiioB/intel-arc-pro-b70-inference-cookbook
Open recipes, engine patches, and benchmark harnesses for LLM inference on Intel Arc Pro B60/B70 (Battlemage, Xe2). MoE 35B at 160 t/s decode / 7.5K t/s prefill single-stream, 27B at 50~ t/s decode / 1.7K t/s prefill single stream. vLLM XPU MTP unlocked. Muse Glimmer recipe added!!
gpustack/gguf-parser-go
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
spark-arena/sparkrun
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD
mozilla-ai/llamafile
Distribute and run LLMs with a single file.
AtomicBot-ai/atomic-agent
Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.
OleksandrChekhovskyi/hax
A minimalist, terminal-native coding agent written in C.
stew675/llama-cpp-rdna-boosts
A set of performant RDNA/HIP patches against llama.cpp focusing on high speed inferencing with numerics assurance
lordmathis/llamactl
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
morganlinton/Albatross
Open source, terminal-first AI coding agent with fully transparent multi-model routing. Local (Ollama, LM Studio, MLX, llama.cpp) or cloud, your keys, one TUI. No black box.
swellweb/reame
CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
sgaofen/humanizer-local-model
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
engeldlgado/toshllm
Run large language models locally on Intel Macs with AMD GPUs - native macOS app with Metal acceleration
Eric-Terminal/ETOS-LLM-Studio
A native LLM client for iOS & Apple Watch. Run local GGUF models offline via llama.cpp, or connect to OpenAI/Claude/Gemini. Features local RAG, Model Context Protocol (MCP) tools, Siri Shortcuts, and cross-device sync. Built with Swift.
julianmb/q38rocm
Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.