Who We Are
An open research, friendly community expanding AI capability at edge.
What We Do
- Build benchmarks and datasets
- Evaluate models with partners
Principles
We prioritize more quality than quantity — minimal overhead, public ilterations.
Latest Releases
→ datasets/seton-labs/bench-easy-6-2026
→ datasets/seton-labs/bench-effortless-6-2026
→ Read our blog
PS: blog posts are short
Why Generalization?
Modern AI performs well on familiar data but struggles with unseen domains. At Seton Labs, out-of-distribution challenges to build systems generalizing beyond training data.
Name Conventions
We use simple and consistent naming syntax.
Difficulty levels: effortless · easy · mid · hard · ultra hard
Each level is based on three factors: number of rows · output size (tokens) · variety of categories
Dataset naming format:
bench-(tier)-(month)-(year)