Neel Nanda - Footnote 3

Note: The claim that the research "Demonstrated that attention mechanisms compose to perform multi-step reasoning" is not explicitly stated in the source. The source mentions that attention heads can compose, but doesn't directly link this to multi-step reasoning. The claim that the research "Provided mathematical descriptions of how models track positional and semantic information" is not explicitly stated in the source. The source discusses how attention heads handle position, but doesn't explicitly state that the research provided mathematical descriptions of how models track positional and semantic information.