superposition

LLMs Can Represent Two Ideas Simultaneously: Evidence for Linear Superposition in Transformers

Averaging two text embeddings lets one pass produce two continuations, showing transformer superposition resto…