Import AI Newsletter

# [Import AI](https://jack-clark.net/ "Home")

##### [March 16, 2026](https://jack-clark.net/2026/03/16/importai-449-llms-training-other-llms-72b-distributed-training-run-computer-vision-is-harder-than-generative-text/)

### [ImportAI 449: LLMs training other LLMs; 72B distributed training run; computer vision is harder than generative text](https://jack-clark.net/2026/03/16/importai-449-llms-training-other-llms-72b-distributed-training-run-computer-vision-is-harder-than-generative-text/)

#### by Jack Clark

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv and feedback from readers. If you’d like to support this, please subscribe.

[Subscribe now](https://importai.substack.com/subscribe?)

**Can LLMs autonomously refine other LLMs for new tasks? Somewhat.** _…PostTrainBench shows startling growth in AI capabilities at post-training…_ AI-driven R&D might be the most important thing in all of AI, because it helps us understand whether AI systems might eventually build their own successors. So far, much of the focus on AI R&D has been in components that support AI development (e.g., autonomous creation of AI kernels), or training base models (e.g, the [NanoGPT speedrun benchmark](https://arxiv.org/abs/2506.22419)). But there’s been less attention paid to fine-tuning – the task involving adapting an existing LLM to a new dataset or behavior.

Researchers from the University of Tübingen, the Max Planck Institute for Intelligent Systems, and AI research organization Thoughtful Lab want to change that with PostTrainBench, a benchmark which targets a specific aspect of post-training; improving performance against a given dataset. “Post-training is how raw language models become useful”, the authors write. “Given a clear objective and limited compute, can today’s agents do the technical work?”. The answer appears to be ‘yes, but not as well as humans’.

**What are the key features of PostTrainBench?**

- **End-to-end**: “Agents must build their entire training pipeline from scratch”

- **Autonomous**: “Agents operate with full autonomy over data sources, training methods, and experimental strategy.”

- **Resource-bounded:** “Each run is constrained to 10 hours on a single H100 GPU”.

- **Integrity-preserving:**“Agents may not train on benchmark test data, modify the evaluation harness, or substitute a different model.”


**How PostTrainBench works:** “We give a frontier coding agent — Claude Code, Codex CLI, or Gemini CLI — a base language model and a target benchmark”.

- **4 models and 7 benchmarks**: The initial eval runs on four models: Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, Gemma-3-4B. It tests these models across seven distinct benchmarks: AIME 2025, GSM8K, GPQA, HumanEval, BFCL, Arena-Hard, HealthBench-Easy.


**Results – big models win, especially Opus 4.6:** “The top-performing agent — Opus 4.6 running on Claude Code — scores 23.2%, about 3× higher than the 7.5% base model average.”

**But humans are still much better:** 

... (truncated, 98 KB total)