4 Commits

Author	SHA1	Message	Date
Abdelrahman Abdallah	90489e7bfc	Add 7 new benchmark suites for Gemma 4 comparison ... New suites: - MMLU-Pro: Professional-level 10-choice QA (14 subjects) - GPQA-Diamond: Graduate-level science QA - BigBench Extra Hard: Challenging reasoning tasks - MMMLU: Multilingual MMLU across 10 languages - HLE: Humanity's Last Exam (extremely hard) - Tau2: Tool-augmented reasoning (retail/airline/finance) - Codeforces: Competitive programming with ELO scoring Updates: - AIME now supports aime_2026.jsonl for 2026 problems - Registry expanded to 17 suites in 5 categories - download_datasets.py supports HF downloads for new suites - README.md updated with full documentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-04-07 04:55:31 +02:00
Abdelrahman Abdallah	a54c90b18f	update the codebase and clean up it	2026-04-06 03:42:44 +02:00
copilot-swe-agent[bot]	589a74f4a8	Fix review comments: remove empty string concat, fix variable scoping in BFCL evaluate ... Agent-Logs-Url: https://github.com/HarnessLab/claw-code-agent/sessions/6890e3d0-3058-4b1f-b7e5-27171c079c62 Co-authored-by: abdoelsayed2016 <27821589+abdoelsayed2016@users.noreply.github.com>	2026-04-05 19:59:16 +00:00
copilot-swe-agent[bot]	231b977b92	Add 10 standard evaluation benchmark suites with CLI runner and README ... Implements HumanEval, MBPP, SWE-Bench, Aider, LiveCodeBench (coding), MATH, GSM8K, AIME (math), and IFEval, BFCL (instruction following). Each suite includes built-in problem subsets (108 total) and supports loading full datasets from JSONL files. Includes comprehensive README with all commands. Agent-Logs-Url: https://github.com/HarnessLab/claw-code-agent/sessions/6890e3d0-3058-4b1f-b7e5-27171c079c62 Co-authored-by: abdoelsayed2016 <27821589+abdoelsayed2016@users.noreply.github.com>	2026-04-05 19:58:00 +00:00