How do transformers work, and why did they replace RNNs?

Question

Accepted Answer

Transformers process a sequence in parallel using self-attention, letting every token look at every other token in one step; they replaced recurrent neural networks because they train far faster on modern hardware and capture long-range dependencies more effectively. As Ashish Vaswani et al., put it on the record: "We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely."

Sources