How DeepSeek rewrote the Transformer
In January 2025, DeepSeek released R1 (and, weeks earlier, V3), competing with o1 and GPT-4o in reasoning — at a fraction of the training and inference cost. The weights were released openly. The trick is not a dataset trick: it is a rewrite of the Transformer’s core, Multi-Head Latent Attention … Read more