#llmops (30 Repositories)
Ranked open-source repositories tagged with #llmops, scored by pull request acceptance likelihood and maintainer engagement velocity.
45.0%
37.4h
30 repositories tagged #llmops
rhesis-ai/rhesis
The collaboration layer for AI teams: domain experts annotate and review agent behavior, engineers improve the agent from what they find.
fkiene/llmtrim
Local proxy that compresses your LLM API requests so you pay less, with no change to the answers. Trims wasted tokens from prompts, history, tool output, and code before they're sent: -31% input / -74% output, measured live. Any provider, no extra model calls. Also an MCP server and embeddable library (Rust, Python, Ruby, Kotlin, Swift, JS/TS).
dynamiq-ai/dynamiq
Dynamiq is an orchestration framework for agentic AI and LLM applications
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Ascend
coze-dev/coze-loop
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.
Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
ferro-labs/ai-gateway
Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.
langfuse/langfuse
🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
0xPlaygrounds/rig
⚙️🦀 Build modular and scalable LLM Applications in Rust
Arize-ai/phoenix
AI Observability & Evaluation
BerriAI/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Giskard-AI/giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
alibaba/rtp-llm
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
bionic-gpt/bionic-gpt
Bionic is sovereign AI for the enterprise — ChatGPT-like AI that runs on-premise and can securely work with your sensitive data and systems.
ComposioHQ/composio
Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
langfuse/langfuse-js
🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework
katanemo/plano
Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
pathwaycom/llm-app
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
taishan666/MaxKB4j
MaxKB4j is an open-source LLMOps platform for LLM workflow applications and RAG developed based on the Java language. The project mainly draws on MaxKB, Dify and FastGPT, and combines the advantages of the two into one project. It is redesigned and developed using the high-performance, high-stability and secure reliable JAVA language.
qiantongtech/qKnow
qKnow is an open-source agent platform for enterprise knowledge intelligence and industry AI. It provides knowledge graphs, RAG, and bot building, encapsulating industry AI into standardized modules for rapid deployment. Supporting unified integration of documents and data, it helps enterprises quickly build knowledge hubs and AI solutions.
zmedelis/bosquet
Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
Nexus-Router/nexus
Govern & Secure your AI
ajndkr/lanarky
The web framework for building LLM microservices [deprecated]
deadbits/vigil-llm
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
raga-ai-hub/RagaAI-Catalyst
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
cloudshipai/station
Station is our open-source runtime that lets teams deploy agents on their own infrastructure with full control.