The Token Is a Group Element: Lie-Algebra Attention over Matrix Lie Groups

arXiv·medium signal

Przemyslaw Musialski reframes a token as an element of a matrix Lie group rather than a feature vector, scoring attention via the parameter-free negative squared algebra norm of the relative pose s_ij = -||log(g_i^-1 g_j)||²/τ. On sequence-completion over SE(2), SO(3), and Aff(2) it matches learned MLP kernels using 50–80x fewer score parameters, while vector-token baselines violate invariance by 5–12 orders of magnitude. It is notable for extending equivariant attention to non-compact, non-abelian affine groups previous methods couldn't handle — relevant to robotics, pose, and geometric ML.