Repository Issues

EleutherAI/lm-evaluation-harness

A framework for few-shot evaluation of language models.

View on GitHub
Stars
 (12,755 stars)
Forks
 (3,306 forks)
Indexed issues
 (24 indexed issues)
open beginner issues
 (0 open beginner issues)
Latest indexed
Jul 17, 2026
Last GitHub push
May 11, 2026
License
MIT License
Contributing guide
Contributing guide
Code of conduct
No code of conduct
Dominant language
Python
PR merge metrics
 (Avg merge 15d 7h) (11 merged PRs in 30d)
Beginner labels
No beginner labels indexed

Issues

23 open indexed issues

RSSAtom
Open
Add TheoremQA
feature requestgood first issuehelp wanted

EleutherAI/lm-evaluation-harness #518 opened May 23, 2023 · Python · 12,755 stars

Why recommendedHas a beginner-friendly label · Contributing guide available
Has a beginner-friendly labelContributing guide available
4 comments0 reactions1 assignee
Open
Implement TyDiQA
feature requestgood first issuehelp wanted

EleutherAI/lm-evaluation-harness #193 opened Jun 10, 2021 · Python · 12,755 stars

Why recommendedNo assignee yet · Has a beginner-friendly label
No assignee yetHas a beginner-friendly labelContributing guide available
3 comments0 reactions0 assignees
Open
Implement XQuAD
feature requestgood first issuehelp wanted

EleutherAI/lm-evaluation-harness #191 opened Jun 10, 2021 · Python · 12,755 stars

Why recommendedNo assignee yet · Has a beginner-friendly label
No assignee yetHas a beginner-friendly labelContributing guide available
3 comments0 reactions0 assignees