2,002 offers
Free Benchmarks & Resources — ranked by value
Agent benchmarks, evals & guides
Scores rate free-tier value on a 1–10 scale. How scores work →
-
promptfoo Benchmarks & Resources 7.7 Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic. 22.9k★1.2k/90d17 devs/90d -
DeepEval Benchmarks & Resources 7.5 The LLM Evaluation Framework 16.6k★601/90d18 devs/90d -
lm-evaluation-harness Benchmarks & Resources 7.0 A framework for few-shot evaluation of language models. 13.2k★51/90d29 devs/90d -
Inspect AI Benchmarks & Resources 7.0 Inspect: A framework for large language model evaluations 2.3k★1.2k/90d24 devs/90d -
awesome-llm-resources Benchmarks & Resources 6.3 🧑🚀 全世界最好的LLM资料总结（多模态生成、Agent、辅助编程、AI审稿、数据处理、模型训练、模型推理、o1 模型、MCP、小语言模型、视觉语言模型） | Summary of the world's best LLM resources. 8.6k★61/90d4 devs/90d -
OSWorld Benchmarks & Resources 6.1 [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments 3k★22/90d18 devs/90d -
OpenAI Evals Benchmarks & Resources 5.8 Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks. 18.8k★2/90d2 devs/90d -
tau2-bench Benchmarks & Resources 5.7 τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains 1.5k★61/90d8 devs/90d -
awesome-web-agents Benchmarks & Resources 5.6 🔥 A list of tools, frameworks, and resources for building AI web agents 1.5k★22/90d13 devs/90d -
Berkeley Function Calling Leaderboard Benchmarks & Resources 5.2 Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls) 12.9k★0/90d0 devs/90d -
Ragas Benchmarks & Resources 4.9 Supercharge Your LLM Application Evaluations 🚀 14.6k★0/90d0 devs/90d -
SWE-bench Benchmarks & Resources 4.6 SWE-bench: Can Language Models Resolve Real-world Github Issues? 5.3k★0/90d0 devs/90d -
Meta Agents Research Environments Benchmarks & Resources 4.5 Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges. 523★9/90d2 devs/90d -
AgentBench Benchmarks & Resources 4.4 A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24) 3.5k★0/90d0 devs/90d -
awesome-mcp-registry Benchmarks & Resources 4.3 AI-curated, self-updating directory of MCP servers. Discovered from GitHub & the official registry, analyzed and ranked by AI weekly. 1★36/90d3 devs/90d -
terminal-bench Benchmarks & Resources 4.2 A benchmark for LLMs on complicated tasks in the terminal 2.4k★0/90d0 devs/90d -
tau-bench Benchmarks & Resources 4.0 Code and Data for Tau-Bench 1.3k★0/90d0 devs/90d -
BrowserGym Benchmarks & Resources 4.0 🌎💪 BrowserGym, a Gym environment for web task automation 1.3k★0/90d0 devs/90d -
Online-Mind2Web Benchmarks & Resources 4.0 An Illusion of Progress? Assessing the Current State of Web Agents 188★7/90d1 devs/90d -
WebArena Benchmarks & Resources 3.8 Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents" 1.5k★0/90d0 devs/90d -
Mind2Web Benchmarks & Resources 3.6 [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web" -- the first LLM-based web agent and benchmark for generalist web agents 1k★0/90d0 devs/90d -
AgentBoard Benchmarks & Resources 2.9 An Analytical Evaluation Board of Multi-turn LLM Agents [NeurIPS 2024 Oral] 423★0/90d0 devs/90d -
AI Agent Benchmark Compendium Benchmarks & Resources 2.6 Compendium of over 50 benchmarks for evaluating AI agents, categorized into Function Calling & Tool Use, General Assistant & Reasoning, Coding & Software Engineering, and Computer Interaction. 168★0/90d0 devs/90d -
GAIA Benchmarks & Resources 2.6 Meta's benchmark of real-world questions testing reasoning, multimodality, web browsing, and tool-use for general assistants. GitHub -
LLM-Agent-Evaluation-Survey Benchmarks & Resources 2.5 Top papers related to LLM-based agent evaluation 96★0/90d0 devs/90d