Arb Research - AI Safety & Forecasting Consulting

web

Arb Research is a small consultancy producing AI safety-adjacent research; their work on AI eval methodology and the AI safety research map (shallowreview.ai) are most directly relevant to the safety community.

Metadata

Importance: 42/100homepage

Summary

Arb Research is a consulting group led by Gavin Leech and Charles Dillon specializing in forecasting, machine learning, and policy research. They produce original research including work on AI evaluation practices, safety research mapping, and technical writing for ML methods. Current focus includes building AI tools and investigating unscientific practices in AI evaluations.

Key Points

•Produces research diagnosing unscientific practices in AI evaluations, directly relevant to AI safety assessment rigor
•Created a map of all AI safety research (shallowreview.ai), a useful meta-resource for the field
•Clients include FAR AI, Open Philanthropy (Coefficient Giving), SCSP, and Schmidt Futures — key EA/AI safety funders
•Works span forecasting methodology, generative biology review, and experiment design for new ML methods
•Authored a major trade book on current AI (Stripe Press: 'Scaling') and PNAS-published technical writing

Cited by 1 page

Page	Type	Quality
Arb Research	Organization	50.0

Cached Content Preview

HTTP 200Fetched Mar 20, 20263 KB

[![](https://arbresearch.com//img/arb-minimal.svg)](https://arbresearch.com/)

[Arb Research](https://arbresearch.com/)

Our consulting work spans
[forecasting](https://www.vox.com/future-perfect/2024/2/13/24070864/samotsvety-forecasting-superforecasters-tetlock),
[machine](https://www.arxiv.org/abs/2602.12413) [learning](https://arxiv.org/abs/2308.10248),
and [policy](https://progress.institute/can-policymakers-trust-forecasters).



We do original research, evidence gathering, and largeish data pipelines.


### Current work


This year, we're building AI tools and investigating unscientific practices in AI evals.


|
|

### [Past work](https://arbresearch.com/work)

|     |     |
| --- | --- |
|  | [A major trade book](https://press.stripe.com/scaling) on current AI |
|  | [Experiment design](https://arxiv.org/abs/2308.10248) and [technical writing](https://www.pnas.org/doi/10.1073/pnas.2415697122) for new ML methods |
|  | [Diagnosing](https://arxiv.org/abs/2407.12220) [unscientific practices](https://www.arxiv.org/abs/2602.12413) in AI evals |
|  | [Annotating](https://frontier2025.netlify.app/) key scientific breakthroughs of our time |
|  | [Location scouting](https://arbresearch.com/files/scouting.pdf) for clinical trials |
|  | [Mapping](https://shallowreview.ai/) all AI safety research |
|  | [Synthesis](https://arxiv.org/abs/2402.04464) and strategy in ten fields at once |
|  | [Reviewing](https://arbresearch.com/files/gen_bio.pdf) recent progress in generative biology |
|  | [Reviewing](https://arbresearch.com/files/comparing_forecasters.pdf) the evidence for generalist forecasters, and implementation [issues](https://progress.institute/can-policymakers-trust-forecasters) |
|  | [Reviewing](https://arbresearch.com/files/elite_education.pdf) the economics of elite education |
|  | [Evaluating](https://www.cold-takes.com/the-track-record-of-futurists-seems-fine/) the forecasting work of Isaac Asimov |

Our clients include
[Stripe](https://press.stripe.com/scaling),
[SCSP](https://www.scsp.ai/),
[Coefficient Giving](https://www.openphilanthropy.org/),
[Schmidt Futures](https://www.schmidtfutures.com/),

[Renaissance Philanthropy](https://www.schmidtfutures.com/),
[the Mercatus Center](https://www.mercatus.org/),
[FAR AI](https://far.ai/),
and [the Institute for Progress](https://progress.institute/).


### [News](https://arbresearch.com/work)

|     |     |
| --- | --- |
| _Feb 2026:_ | [Semantic duplicates confound apparent AI progress](https://arbresearch.com/work#Semantic%20duplicates%20confound%20apparent%20AI%20progress) |
| _Dec 2025:_ | [2025 review](https://arbresearch.com/work#2025%20review) |
| _Dec 2025:_ | [Breakthroughs of the year](https://arbresearch.com/work#Breakthroughs%20of%20the%20year) |

### [Express interest](mailto:hi@arbresearch.com)

by
[Gavin Leech](https://www.gleech.org/cv.pdf)
&
[Charles Dillon](https://substack.com/@charlesd353)

Resource ID: 3f6db916d9b25357 | Stable ID: NmZlZjdjMD