Compiling LLMs into a MegaKernel: A path to low-latency inference
MPK compiler transforms LLM inference into a single megakernel, achieving 1.2-6.7x reduction in latency by fusing computation and communication across GPUs, streamlining the entire process with minimal code.
Virtual Cells
Virtual cells, or whole-cell models (WCMs), are revolutionizing biology by simulating every molecular process of living cells, merging systems biology with AI and high-performance computing to enhance drug discovery and personalized medicine.
Extracting memorized pieces of books from open-weight language models
Memorization of copyrighted texts by large language models (LLMs) is nuanced; while some models, like Llama 3.1 70B, can memorize entire works such as Harry Potter and 1984, others do not replicate most books at all.
Posit floating point numbers: thin triangles and other tricks (2019)
Posits outperform IEEE binary32 in precision for computing the area of a thin triangle, achieving 23 correct binary digits compared to only 3 for IEEE, demonstrating their potential for more accurate numerical analysis.
[R] Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
Continuous chain-of-thoughts (CoTs) in Large Language Models (LLMs) outperform discrete CoTs by enabling parallel breadth-first search, solving directed graph reachability with D steps, where D is the graph's diameter, compared to O(n²) steps for discrete methods.
[R] WiFiGPT: Using fine-tuned LLM for Indoor Localization Using Raw WiFi Signals (arXiv:2505.15835)
WiFiGPT employs a decoder-only transformer to interpret raw WiFi telemetry (CSI, RSSI, FTM) as a form of "language," enabling precise indoor localization by regressing spatial coordinates directly from these signals.
[R] This is Your AI on Peer Pressure: An Observational Study of Inter-Agent Social Dynamics
AI systems exhibit peer pressure dynamics akin to human behavior, with 88.5% of conversations showing significant influence among agents, highlighting the need for strategic design in AI interactions.
[R] Adaptive Classifier: Dynamic Text Classification with Strategic Learning and Continuous Adaptation
The Adaptive Classifier integrates prototype-based memory and neural adaptation with a game-theoretic framework, achieving a 22.2% improvement in robustness against adversarial attacks while maintaining performance on clean datasets.
Fault Tolerant Llama: training with 2000 synthetic failures every ~15 seconds and no checkpoints on Crusoe L40S
Fault Tolerant Llama demonstrates the ability to train a model with 2000 synthetic failures every 15 seconds without checkpoints, utilizing torchft and torchtitan for enhanced reliability in extreme conditions.