Automated Researchers Can Subtly Sandbag

Automated Researchers Can Subtly Sandbag 
 
 
 
 
 
 
 
 
 

 
 Alignment Science Blog 
 
 
 Automated Researchers Can Subtly Sandbag

 

 
 
 
 tl;dr 
 When prompted, current models can sandbag ML experiments and research decisions without being detected by
 zero-shot prompted monitors. Claude 3.5 Sonnet (new) can only sandbag effectively when seeing a one-shot
 example, while Claude 3.7 Sonnet can do this without an example (zero-shot). We are not yet worried about
 sabotage in today's models, as the sandbagging we observe today would be noticed by humans.

 
 
 AI models might soon assist with large parts of AI and AI safety research, completing multiple hours of work
 independently. These automated researchers will likely not be sufficiently capable to cause immediate
 catastrophic harm. However, they will have the opportunity to engage in sabotage , including
 sabotaging the research we rely on to make more powerful AI models safe. For example, a malicious automated
 researcher might slow down AI safety research by purposefully making promising experiments fail.

 In this work we aim to ensure that automated researchers are not  sabotaging research without detection. It may be hard to
 determine whether a powerful model is aligned or simply faking
 alignment . We instead pessimistically assume that an automated researcher is actively misleading our
 research. We defend against the automated researcher’s sabotage by adding a second model that monitors the
 automated researcher’s outputs for signs of manipulation. We then explore which strategies the saboteur
 could use to bypass these defenses, and how good current models are at implementing these strategies.

 We use a strong, untrusted model to act as the potentially malicious automated researcher. This simulates a
 worst-case scenario where the alignment of a deployed state-of-the-art model is compromised despite
 pre-deployment testing. If we use the misaligned automated researcher to monitor and flag flawed research,
 it might sabotage these predictions as well, so we use weaker models as monitors. Weaker models are more
 trustworthy because they stem from an earlier, time-tested generation of models and because misalignment and
 alignment faking might only emerge with greater model capabilities. Our approach is similar to recent   work   on   AI
 Control .

 We study two broad classes of strategies a malicious AI system could use to sabotage research: 

 
 In experiment sandbagging , a malicious
 automated researcher tries to manipulate experiments to slow down a targeted area of safety-relevant
 research. 

 In research decision steering , the
 automated researcher aims to manipulate arguments and discussion to favor a certain machine learning
 (ML) solution. For example, it may exaggerate one interpretation of an experiment and downplay the
 other.

 
 For experiment sandbagging, we let the automated researcher implement a solution to ML tasks by writing and
 executing Python code, si

... (truncated, 98 KB total)