New York University — LLM Cybersecurity Benchmark

[Navigating Transformative AI] Open Philanthropy recommended a grant of $2,077,350 to New York University to support a project to develop a benchmark for large language model (LLM) agent cyber operations capabilities, in partnership with New Foundry. The project will be led by Alex Leader, Dan Zhao, and He He.  This grant was funded via a request for proposals for projects benchmarking LLM agents on consequential real-world tasks and falls within our focus area of potential risks from advanced artificial intelligence.