Linformer
Linformer is a neural network architecture designed to reduce the computational complexity of the Transformer model by approximating self-attention with low-rank projections. It enables efficient processing of long sequences in natural language processing and other tasks.