Softmax Attention

Retrieval-Oriented Deep Learning in Expanding Nonparametric Neural Networks

The paper reframes linear-layer training through the duality between gradient descent and linear attention, th…