Transformers articles

4 articles about Transformers, newest first.

• Optimizing RMS Norm with FlashNorm and CUDA Streams — Learn how Filip Makraduli uses weight folding and CUDA stream synchronization to speed up RMS Norm in transformers by up to 35% while avoiding race conditions.
• AI Autocomplete for Piano: 108 Notes Per Second on an iPhone — A 125M‑parameter transformer can autocomplete piano performances in real time, hitting 108 notes per second on an iPhone 15, showcasing AI's creative potential.
• Metis: The New AI Transformer That Actually Remembers — Meet Metis, a groundbreaking transformer architecture that integrates memory directly into its attention mechanism, ending the need for external memory files.
• Attention Is All You Need: The Paper That Changed AI Forever — Discover how the Transformer architecture replaced old AI models and paved the way for ChatGPT and the modern LLM revolution.