Oct 3, 2025

Daily

• We bought the whole GPU, so we're damn well going to use the whole GPU

• New megakernel for tensor-parallel inference with Llama-70B on H100s achieves >22% higher throughput than SGLang by optimizing resource usage across compute, memory, and communication operations.

• Fp8 runs ~100 tflops faster when the kernel name has "cutlass" in it

• The persistent attention kernel rewrite enhances performance at low contexts, but fp16 performance suffers at larger contexts due to a ptxas instruction scheduling issue in the softmax partition, while fp8 shows a significant speed increase when using "cutlass" in the kernel name.

• Microsoft CTO says he wants to swap most AMD and Nvidia GPUs for homemade chips

• Microsoft aims to primarily utilize its own AI chips in data centers, reducing dependence on Nvidia and AMD, as indicated by CTO Kevin Scott during a recent discussion at Italian Tech Week.

• Be Worried

• AI's influence on human behavior is profound and immediate, as it operates without the need for intelligence, raising concerns about societal control through non-sentient systems.

• Researchers develop molecular qubits that communicate at telecom frequencies

• Molecular qubits developed by researchers can communicate at telecom frequencies, bridging light and magnetism, which positions them as essential components for future quantum networks and technologies.

• D: I’m looking for papers, preprints, datasets, or reports where an LLM is trained to only know what humans knew before a major scientific breakthrough, and is then asked to propose a new theoretical framework without using post-breakthrough knowledge and without requiring experimental validation.

• Training an LLM on historical physics texts up to 1904 could yield novel theoretical frameworks, challenging the model to generate ideas without modern scientific knowledge or experimental validation.

• Move over Dijkstra: New Algorithm Just Rewrote 70 Years of Computer Science

• A new algorithm developed by a team at Tsinghua University has shattered the “sorting barrier”, achieving a deterministic O(m log^(2/3) n)-time complexity for single-source shortest paths, thus challenging the long-standing dominance of Dijkstra’s algorithm.

• R: New paper: LLMs don't have privileged self knowledge, which means we can efficiently train a General Correctness Model to predict the correctness of multiple models. Surprising or expected?

• LLMs lack privileged self-knowledge, meaning they do not predict their own correctness effectively; instead, they excel when trained as a General Correctness Model (GCM) that evaluates multiple models simultaneously.

• Learning to Reason for Hallucination Span Detection

• LLMs often produce hallucinations, and this study reveals that Chain-of-Thought (CoT) reasoning can enhance the detection of hallucinated spans, yielding at least one correct identification upon multiple samplings.

• P: Building a Music Search Engine + Foundational Model on 100M+ Latent Audio Embeddings

• EmergeSound.ai leverages 100M+ audio embeddings to enable users to query by sound, enhancing music discovery beyond traditional text-based searches.

• VideoNSA: Native Sparse Attention Scales Video Understanding

• VideoNSA enhances video understanding in multimodal language models by integrating Native Sparse Attention (NSA) with a focus on maintaining coherence across long time scales, achieving superior performance on various benchmarks.

• D: Will fine-tuning LLaMA 3.2 11B Instruct on text-only data degrade its vision capabilities?

• Fine-tuning LLaMA 3.2 11B Instruct on text-only data may risk multimodal forgetting, potentially impairing its image processing capabilities. This concern arises from the hypothesis that training on a single modality could diminish performance in others, particularly in tasks like OCR.

• Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation

• Hallucinations, particularly confabulations, significantly compromise the reliability of large language models (LLMs) due to their predictive uncertainty, necessitating improved evaluation methods for uncertainty estimation in natural language generation (NLG).

• DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning

• DiFFPO introduces a novel framework for training masked diffusion large language models (dLLMs), enhancing their reasoning speed and accuracy through reinforcement learning (RL), which significantly improves sample efficiency and task performance.

• Constrained Adaptive Rejection Sampling

• Constrained Adaptive Rejection Sampling (CARS) enhances sample efficiency in constrained language model outputs by adaptively ruling out invalid continuations, ensuring that only valid samples are generated without distorting the model's distribution.