zk-data-agent

Files

T

Abdelrahman Abdallah 90489e7bfc Add 7 new benchmark suites for Gemma 4 comparison

New suites:
- MMLU-Pro: Professional-level 10-choice QA (14 subjects)
- GPQA-Diamond: Graduate-level science QA
- BigBench Extra Hard: Challenging reasoning tasks
- MMMLU: Multilingual MMLU across 10 languages
- HLE: Humanity's Last Exam (extremely hard)
- Tau2: Tool-augmented reasoning (retail/airline/finance)
- Codeforces: Competitive programming with ELO scoring

Updates:
- AIME now supports aime_2026.jsonl for 2026 problems
- Registry expanded to 17 suites in 5 categories
- download_datasets.py supports HF downloads for new suites
- README.md updated with full documentation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-04-07 04:55:31 +02:00

__init__.py

Add 10 standard evaluation benchmark suites with CLI runner and README

2026-04-05 19:58:00 +00:00

aider.py

Add 10 standard evaluation benchmark suites with CLI runner and README

2026-04-05 19:58:00 +00:00

aime.py

Add 7 new benchmark suites for Gemma 4 comparison

2026-04-07 04:55:31 +02:00

base.py

update the codebase and clean up it