Watermarking language models

This groundbreaking paper introduces a sophisticated watermarking method for large language models that addresses critical challenges in AI-generated text detection. The core innovation is a 'soft' watermarking technique that probabilistically promotes certain tokens during text generation, creating a statistically detectable signature without significantly degrading text quality. The methodology involves selecting a randomized set of 'green' tokens and subtly biasing the language model's sampling towards these tokens. This approach is particularly powerful because it works across different sampling strategies like multinomial sampling and beam search, and can be implemented with minimal impact on text perplexity. The authors provide rigorous theoretical analysis, demonstrating how the watermark's detectability relates to the entropy of generated text, and present comprehensive empirical validation using the OPT model family.