#evals (21 Repositories)
Ranked open-source repositories tagged with #evals, scored by pull request acceptance likelihood and maintainer engagement velocity.
38.3%
69.1h
21 repositories tagged #evals
ShenSeanChen/waku-agent
Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows.
truera/trulens
Evaluation and Tracking for LLM Experiments and AI Agents
hud-evals/hud-python
RL environments + evals for AI agents. Define once, train anything.
harbor-framework/harbor
Framework for evaluating and improving agents
benchflow-ai/benchflow
Research infra for creating RL environments, post-training, and evals.
AgentEvalHQ/AgentEval
AgentEval is the comprehensive .NET toolkit for AI agent evaluation—tool usage validation, RAG quality metrics, stochastic evaluation, and model comparison—built first for Microsoft Agent Framework (MAF) and Microsoft.Extensions.AI. What RAGAS, PromptFoo and DeepEval do for Python, AgentEval does for .NET
tikalk/adlc-team-skills
Agent skills for the Agentic SDLC: team lifecycle (team-boot, team-learn, team-init, team-repair), software factory, evals, CDR lifecycle with confidence scoring, and ADR/PDR/ChDR decision records. Works with Claude Code, Codex, OpenCode, Cursor, Copilot, and more.
minghinmatthewlam/openbench
Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
prime-radiant-inc/superpowers-evals
Behavioral eval lab (Quorum) for the superpowers project that drives real coding-agent CLIs (Claude, Codex, Gemini, Kimi, and more) through a QA agent and grades them on workflow compliance against scenario criteria and deterministic post-checks.
Jwuthri/Tracely
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
Jwuthri/Tracely-ai
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
NiceEval/NiceEval
build eval for your agent in 10 mins
calmrocks/ai-engineer-notebooks
Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.
LilMGenius/paperthin
Low-level agentic design patterns. Turning old engineering wisdom into reflexes your agent reaches for on its own—on any agent.
edonadei/caliper
Run your real agent with and without your skills, MCPs, and rules. See which ones actually help, and what they cost in tokens. Supports Claude Code, Codex, Pi, and Hermes.
adewale/skill-eval-harness
Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters
overmind-core/overmind
The platform for continuously improving AI agents.
METR/vivaria
Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
spences10/my-pi
Composable Pi coding agent with MCP, LSP, agent chains, prompt presets, and local eval telemetry
GitHamza0206/simba
OpenSource Production ready Customer service with built in Evals and monitoring
mattpocock/evalite
Evaluate your LLM-powered apps with TypeScript