Diamond Thermal Conductivity: A New Era in Chip Cooling
Diamond's exceptional thermal conductivity—up to 2,400 watts per meter per kelvin—is now harnessed in chip technology, allowing for significant heat dissipation and improved performance in high-density electronics.
Why can't transformers learn multiplication?
Transformers struggle with multi-digit multiplication due to their inability to effectively learn and utilize long-range dependencies, despite evidence that they can encode these structures through attention mechanisms.
How was Multi-head Latent Attention not a thing before DeepSeek-V2 came up with it?
Multi-head Latent Attention (MLA), introduced by DeepSeek-V2 in 2024, projects keys and values into a latent space, significantly reducing computational complexity in attention mechanisms.
ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
ChunkLLM introduces a lightweight and pluggable framework that enhances inference speed for large language models (LLMs) by utilizing QK Adapters and Chunk Adapters to optimize attention mechanisms and chunk detection.
Startup plans to cool data centers by converting heat to light
Photonic cooling technology by Maxwell Labs aims to revolutionize chip cooling by converting heat into light, allowing for targeted cooling of hot spots with laser precision, potentially eliminating the dark silicon problem that limits chip performance.
Compiler optimizations for 5.8ms GPT-OSS-120B inference (not on GPUs)
Performance of gpt-oss-120b was achieved at 5.8 ms per output token using two RNGD cards, showcasing significant power efficiency under 180 W per card, enabled by advanced compiler optimizations and hardware integration.
Torchcomms: A modern PyTorch communications API
Torchcomms is an experimental communication API for PyTorch Distributed, designed to support over 100,000 GPUs and facilitate rapid prototyping of communication primitives, enhancing scalability and flexibility in distributed training.
Signal Processing for AI — A New Way to Think About LLMs and ANN Search
Signal processing principles can enhance AI models and embedding spaces, leading to improved efficiency and accuracy in handling noisy data, as demonstrated in collaboration with Prof. Gunnar Carlsson from Stanford.
UFIPC: Physics-based AI Complexity Benchmark - Models with identical MMLU scores differ 29% in complexity
Models with identical MMLU scores can differ by 29% in architectural complexity, highlighting the need for benchmarks like UFIPC that assess AI beyond mere task accuracy.