Long-Horizon Autonomous Tasks

Anthropic introduces Constitutional AI (CAI), a method for training AI systems to be harmless using a set of principles (a 'constitution') and AI-generated feedback rather than relying solely on human labelers. The approach uses a two-phase process: supervised learning from AI self-critique and revision, followed by reinforcement learning from AI feedback (RLAIF). This reduces dependence on human red-teaming for harmful content while maintaining helpfulness.

This paper introduces Retrieval-Augmented Generation (RAG), a framework that combines parametric memory (pre-trained language models) with non-parametric memory (a dense vector index of Wikipedia) to improve performance on knowledge-intensive NLP tasks. RAG models retrieve relevant documents at inference time and condition text generation on them, achieving state-of-the-art results on open-domain QA benchmarks while enabling knowledge updates without full retraining.

The Alignment Research Center (ARC) is a non-profit research organization focused on technical AI alignment and safety research. ARC works on understanding and addressing risks from advanced AI systems, including interpretability, evaluations, and identifying dangerous AI capabilities before deployment.

This paper introduces the 'AI control' framework, which focuses on ensuring AI systems behave safely even if they are deceptively misaligned or actively trying to subvert oversight. It proposes evaluation protocols and mechanisms to maintain safety against intentional subversion by advanced AI models, treating safety as a red-team/blue-team problem between AI and human overseers.

MemGPT addresses the fundamental limitation of LLMs' finite context windows by implementing virtual context management, inspired by hierarchical memory systems in operating systems. The system intelligently manages multiple memory tiers to provide the appearance of extended context, enabling LLMs to process documents far larger than their native context window and maintain coherent long-term conversations. The authors demonstrate MemGPT's effectiveness in document analysis and multi-session chat applications, where it enables conversational agents to remember, reflect, and evolve through extended interactions.

This paper introduces h-DQN, a hierarchical deep Q-network framework that combines two-level value functions operating at different temporal scales with intrinsic motivation to tackle sparse-reward exploration. A top-level controller sets subgoals while a lower-level controller learns primitive actions to achieve them, enabling more efficient exploration. The approach achieves notable results on Montezuma's Revenge, a benchmark known for extremely sparse rewards.

This foundational 2015 MIRI paper by Soares, Fallenstein, Yudkowsky, and Armstrong introduces the formal concept of 'corrigibility'—the property of an AI system that cooperates with corrective interventions despite rational incentives to resist shutdown or preference modification. The paper analyzes utility function designs for safe shutdown behavior and finds no proposal fully satisfies all desiderata, framing corrigibility as an open research problem.

SWE-bench is a benchmark and leaderboard platform for evaluating AI models on real-world software engineering tasks, particularly resolving GitHub issues in open-source Python repositories. It offers multiple dataset variants (Lite, Verified, Multimodal) and standardized metrics to compare coding agents. It has become a widely-used standard for assessing the practical software engineering capabilities of LLM-based agents.

CNAS is a Washington D.C.-based national security think tank publishing research on defense, technology policy, economic security, and AI governance. Its Technology & National Security program produces policy-relevant work on AI, cybersecurity, and emerging technologies with implications for AI safety and governance.

Transformer-XL addresses the fixed-length context limitation of standard Transformers in language modeling by introducing segment-level recurrence and a novel positional encoding scheme. This architecture enables models to learn dependencies 450% longer than vanilla Transformers while maintaining temporal coherence and avoiding context fragmentation. The approach achieves state-of-the-art results across multiple benchmarks (enwik8, text8, WikiText-103, etc.), demonstrates 1,800+ times faster evaluation speed, and can generate coherent text spanning thousands of tokens.

This paper proposes 'debate' as a scalable oversight mechanism for training AI systems on complex tasks that are difficult for humans to directly evaluate. Two agents compete in a zero-sum debate game, taking turns making statements about a question or proposed action, after which a human judge determines which agent provided more truthful and useful information. The authors draw an analogy to complexity theory, arguing that debate with optimal play can answer questions in PSPACE with polynomial-time judges (compared to NP for direct human judgment). They demonstrate initial results on MNIST classification where debate significantly improves classifier accuracy, and discuss theoretical implications and potential scaling challenges.

ReAct introduces a prompting paradigm that interleaves reasoning traces with task-specific actions in LLMs, enabling them to use external tools (e.g., Wikipedia API) while reasoning. This approach reduces hallucination and error propagation compared to chain-of-thought alone, and outperforms imitation/reinforcement learning baselines on interactive decision-making benchmarks by large margins.

MIRI is a nonprofit research organization focused on ensuring that advanced AI systems are safe and beneficial. It conducts technical research on the mathematical foundations of AI alignment, aiming to solve core theoretical problems before transformative AI is developed. MIRI is one of the pioneering organizations in the AI safety field.

Cursor is an AI-powered code editor built on VS Code that features an 'Agent Mode' enabling autonomous multi-step coding tasks. It allows AI agents to plan, write, edit, and execute code across a codebase with minimal human intervention. This represents a practical deployment of agentic AI systems in software development workflows.

CHAI is a UC Berkeley research center dedicated to reorienting AI development toward systems that are provably beneficial and aligned with human values. It conducts technical and conceptual research on problems including value alignment, corrigibility, and AI safety, and serves as a major hub for academic AI safety work.

This paper introduces Imitation learning from Language Feedback (ILF), a method for aligning language models with human preferences using natural language feedback rather than just pairwise comparisons. ILF operates iteratively by conditioning the model on inputs, initial outputs, and feedback to generate refinements, selecting the best refinement, and finetuning to maximize its likelihood. The authors provide theoretical grounding by connecting ILF to Bayesian inference and demonstrate through experiments on summarization tasks that ILF effectively incorporates feedback, scales well with dataset size, and can outperform finetuning on human-written summaries. Combining language and comparison feedback yields the best results, achieving human-level performance.

Cognition Labs introduces Devin, an autonomous AI agent capable of end-to-end software engineering tasks including writing, debugging, and deploying code. Devin represents a significant capabilities milestone demonstrating long-horizon task completion with persistent memory and tool use. The announcement highlights performance on SWE-bench and showcases Devin completing real engineering jobs autonomously.

Tree of Thoughts (ToT) is a novel inference framework that extends Chain of Thought prompting by enabling language models to explore multiple reasoning paths and perform deliberate decision-making with lookahead and backtracking capabilities. Rather than following a single left-to-right token generation process, ToT treats intermediate reasoning steps as coherent units of text (thoughts) that can be evaluated and explored systematically. The framework significantly improves performance on complex tasks requiring planning and search, achieving 74% success on Game of 24 compared to 4% for GPT-4 with standard chain-of-thought prompting.

This paper challenges the standard two-phase RLHF paradigm by arguing that human preferences follow regret under optimal policies rather than reward distributions. The authors propose Contrastive Preference Learning (CPL), a new algorithm that learns optimal policies directly from human preferences without explicitly learning reward functions, thereby avoiding the optimization challenges of traditional RL. CPL uses a simple contrastive objective and is fully off-policy, enabling it to scale to high-dimensional and sequential decision-making problems while being simpler than existing RLHF methods.

Perplexity AI is an AI-powered search and answer engine that uses large language models to provide cited, conversational responses to queries. It combines web search with generative AI to deliver synthesized information with source citations. The Pro version offers enhanced research capabilities including deeper search and access to more powerful models.

WebArena is a benchmark environment for evaluating autonomous web-browsing AI agents on realistic, long-horizon tasks across functional websites (e-commerce, forums, code repos, etc.). It tests agents' ability to complete complex multi-step goals requiring planning, navigation, and tool use in a self-hosted web ecosystem. The benchmark helps measure progress and identify limitations in agentic AI systems operating in realistic digital environments.

This foundational paper by Amodei et al. identifies five practical AI safety research problems: avoiding side effects, avoiding reward hacking, scalable oversight, safe exploration, and robustness to distributional shift. It frames these as concrete technical challenges arising from real-world ML system design, providing a research agenda that has significantly shaped the field of AI safety.

RAND Corporation's AI research hub covers policy, national security, and governance implications of artificial intelligence. It aggregates reports, analyses, and commentary on AI risks, military applications, and regulatory frameworks from one of the leading U.S. defense and policy think tanks.

AgentBench is a comprehensive multi-dimensional benchmark designed to evaluate Large Language Models (LLMs) as autonomous agents across 8 distinct interactive environments. The study evaluates both API-based and open-source LLMs, revealing significant performance gaps between top commercial models and open-source alternatives up to 70B parameters. The research identifies key failure modes—poor long-term reasoning, weak decision-making, and inadequate instruction following—and proposes that improvements in instruction following and high-quality multi-round alignment training could enhance agent performance. Notably, the findings challenge conventional assumptions about code training's universal benefits for agent tasks.

This paper addresses the fundamental challenge in model-based reinforcement learning of balancing data efficiency gains from learned models against the bias introduced by model-generated data. The authors provide theoretical analysis of model usage in policy optimization, showing that a simple approach of generating short rollouts from learned models branched off real data can achieve both improved sample efficiency over prior model-based methods and asymptotic performance matching state-of-the-art model-free algorithms. They demonstrate that incorporating empirical estimates of model generalization into theoretical guarantees justifies model usage, and their method scales effectively to longer horizons where other model-based approaches fail.

This is OpenAI's research overview page describing their work toward artificial general intelligence (AGI). The page outlines OpenAI's mission to ensure AGI benefits all of humanity and highlights their major research focus areas: the GPT series (versatile language models for text, images, and reasoning), the o series (advanced reasoning systems using chain-of-thought processes for complex STEM problems), visual models (CLIP, DALL-E, Sora for image and video generation), and audio models (speech recognition and music generation). The page serves as a hub linking to detailed research announcements and technical blogs across these domains.

This paper addresses automatic science journalism—converting technical scientific papers into accessible news articles for general audiences. The authors introduce SciTechNews, a new dataset of scientific papers paired with corresponding news articles and expert summaries, and propose a technical framework that leverages paper discourse structure and metadata to guide generation. Their approach outperforms baselines like Alpaca and ChatGPT in creating meaningful content plans, simplifying information, and producing coherent layman-friendly reports.

DeepMind's safety research homepage outlines the organization's approach to ensuring AI systems are safe, beneficial, and aligned with human values. It covers their research priorities, teams, and published work across technical safety, alignment, and responsible deployment.

The Centre for the Governance of AI (GovAI) is a leading research organization dedicated to helping decision-makers navigate the transition to a world with advanced AI. It produces rigorous research on AI governance, policy, and societal impacts, while fostering a global talent pipeline for responsible AI oversight. GovAI bridges technical AI safety concerns with practical policy recommendations.

Anthropic's research page aggregates their work across AI alignment, mechanistic interpretability, and societal impact assessment, all oriented toward understanding and mitigating risks from increasingly capable AI systems. It serves as a central hub for their published findings and ongoing safety-focused investigations.

METR presents empirical research showing that AI models' ability to complete increasingly long autonomous tasks is growing exponentially, with the maximum task length that models can successfully complete roughly doubling every 7 months. This 'task length' metric serves as a practical proxy for measuring real-world AI capability progression and agentic autonomy.

OpenAI collaborated with human software developers to audit and filter the original SWE-bench benchmark, removing problematic or ambiguous test samples to create SWE-bench Verified. This improved benchmark provides more reliable and fair evaluations of AI models' ability to solve real-world software engineering tasks. It addresses concerns that inflated or misleading scores on the original benchmark obscured true model capabilities.

A McKinsey practitioner-oriented guide for technology leaders on safely deploying agentic AI systems in enterprise contexts. The resource likely covers risk frameworks, security considerations, and governance practices for AI agents that can take autonomous actions. Content is inaccessible due to access restrictions.

Anthropic announces Claude Opus 4.5, their most capable model optimized for coding, agentic tasks, and computer use, with significantly reduced pricing ($5/$25 per million tokens). The model demonstrates state-of-the-art performance on software engineering benchmarks, long-horizon autonomous tasks, and multi-step reasoning while being notably more token-efficient than predecessors.

Anthropic announces a major capability expansion: Claude 3.5 Sonnet gains 'computer use' ability (controlling mouse, keyboard, and screen), an upgraded Claude 3.5 Sonnet with improved reasoning and coding, and the fast/affordable Claude 3.5 Haiku. Computer use represents a significant step toward agentic AI that can autonomously operate computers to complete tasks.

SWE-bench Pro is a rigorous benchmark by Scale AI that evaluates AI agents on real-world software engineering tasks drawn from both public and private repositories. It addresses limitations of existing benchmarks by emphasizing realistic, challenging problem-solving scenarios. The leaderboard tracks and compares performance of leading AI coding agents.

The NIST AI RMF is a voluntary, consensus-driven framework released in January 2023 to help organizations identify, assess, and manage risks associated with AI systems while promoting trustworthiness across design, development, deployment, and evaluation. It provides structured guidance organized around core functions and is accompanied by a Playbook, Roadmap, and a Generative AI Profile (2024) addressing risks specific to generative AI systems.

This paper introduces a new metric called '50%-task-completion time horizon' to measure AI capabilities in human-relatable terms—specifically, the time humans with domain expertise typically need to complete tasks that AI models can solve with 50% success rate. The authors evaluated frontier models like Claude 3.7 Sonnet on a dataset combining existing benchmarks and 66 novel tasks, finding current models achieve approximately 50 minutes on this metric. Notably, the AI time horizon has doubled roughly every seven months since 2019, driven primarily by improvements in reliability, error adaptation, logical reasoning, and tool use. If this trend continues, the authors project that within 5 years, AI systems could automate many software tasks currently requiring a month of human effort.