Attention Mechanism

An attention mechanism is a neural-network operation that lets a model compute a weighted combination of values based on learned similarities between queries and keys — in plain terms, it lets the network decide which parts of its input to focus on for each output position. Attention was introduced in the context of sequence-to-sequence machine translation (Bahdanau et al., 2014) as a fix for the bottleneck of encoding an entire sentence into a single fixed-size hidden state. It became the dominant mechanism in AI six years later when the 2017 Transformer paper showed that a network built entirely from attention — no recurrence, no convolution — could match or exceed recurrent networks across NLP tasks.

The modern attention taxonomy has several axes. **Self-attention** — queries, keys, and values all come from the same sequence — is the operation inside every Transformer block. **Cross-attention** — queries from one sequence, keys and values from another — is how encoder-decoder Transformers fuse input and output. **Multi-head attention** — multiple parallel attention operations on different learned projections — lets the network attend to different relationship types simultaneously. **Causal (masked) attention** restricts each position to attending only to earlier positions, which is what makes decoder-only LLMs generate tokens left-to-right. **Grouped-query** and **multi-query** attention variants share keys and values across heads to reduce inference memory.

For APAC mid-market teams, attention is rarely a design choice at the architecture level — you inherit it from whichever pretrained model you adopt. What matters is understanding its cost structure: attention is **O(n²) in sequence length** in both memory and compute during training, and a linear factor during autoregressive decoding (each new token attends to all previous tokens). This is why long context windows require algorithmic innovations (FlashAttention for memory efficiency, sliding-window and sparse attention for sub-quadratic cost, state-space alternatives like Mamba).

The non-obvious operational note: **attention patterns look interpretable but are not reliable explanations**. It is tempting to say "the model attended strongly from position i to position j, so j caused the output at i". Interpretability research (activation patching, circuit analysis) shows attention patterns are suggestive but incomplete — the residual stream, feed-forward layers, and attention all interact. Use attention visualisations as a debugging hint, not as a definitive explanation of behaviour.