The Wall as Unpaid Synchronization Tax
The wall marks the boundary where the synchronization tax goes unpaid: where gradient descent has never supplied the energy needed to rotate each position's hidden state into alignment with the unembedding
The wall marks the boundary where the synchronization tax goes unpaid: where gradient descent has never supplied the energy needed to rotate each position's hidden state into alignment with the unembedding