transformer interpretability

Paragraph Boundaries Are More Than Whitespace: Compression Depth Reveals Hierarchical Structure

Hierarchical RoPE tests paragraph structure; deeper cross-paragraph attention compression separates meaningful…