LLM Inference articles

3 articles about LLM Inference, newest first.

• Superlinked: Scaling a Fleet of Small Open-Source Models — Learn how Superlinked solves the infrastructure bottlenecks of serving multiple small AI models using a pull-based queue and a Rust sidecar architecture.
• Debugging vLLM: How AI21 Fixed Two Silent Jamba Model Bugs — Learn how AI21 engineers used logprob forensics and memory pressure to find two critical bugs in vLLM affecting the hybrid Jamba model architecture.
• Optimizing RMS Norm with FlashNorm and CUDA Streams — Learn how Filip Makraduli uses weight folding and CUDA stream synchronization to speed up RMS Norm in transformers by up to 35% while avoiding race conditions.