Inkling is a Mixture-of-Experts transformer model with 975 billion total parameters and 41 billion active, designed for multimodal reasoning across text, images, and audio, and is available for customization on Tinker.
Kimi K3: Open Frontier Intelligence
Kimi K3 is a groundbreaking 2.8 trillion-parameter model that integrates advanced Kimi Delta Attention and Attention Residuals, enabling it to excel in long-horizon coding and knowledge work while being the first open model in the 3T-class category.
German AI consortium releases Soofi S, an open 30B model that tops benchmarks
Soofi S, an open-source language model developed by a German consortium, utilizes a hybrid architecture that activates only 3.2 billion of its 31.6 billion parameters per token, ensuring efficient processing even with lengthy inputs.
Schema Harness Achieves ~99% on Arc‑AGI‑3 Public
Schema achieves ~99% RHAE on ARC-AGI-3 Public set, demonstrating a significant leap in action efficiency by integrating state grounding and mechanism discovery into a single editable program, allowing for real-time model adjustments based on observations.
Detecting LLM-Generated Texts with “Classical” Machine Learning
Mainstream LLM-generated text can be distinguished from human-written content with approximately 85% accuracy using traditional machine learning models like Linear SVC and Naive Bayes. This detection leverages statistical patterns inherent in AI-generated text, which are often consistent and detectable compared to the variability of human writing.
Mechanistic interpretability: a first paper on disentangling a convolutional neuron
The study reveals that the Hadamard product of a neuron's receptive field and weights effectively represents what the neuron detects, allowing for the clustering of patterns like cars, cats, and dogs into clean monosemantic groups, alongside unexpected clusters such as letters and human faces.
Designing APIs for Agents
Designing APIs for agents requires a shift from human-centric principles, emphasizing clarity and explicitness to accommodate AI's ability to process extensive documentation and generate code rapidly.
High-Bandwidth Flash offers efficient storage for model weights
High-Bandwidth Flash technology enables dramatically improved read speeds by stacking flash memory dies, addressing the storage limitations faced by AI models.
Pseudpocalypse
Pseudpocalypse suggests that unique writing styles can be statistically linked to authors, potentially compromising pseudonymity as machine learning advances in text analysis.
PyTorch model running 170x slower on T4 vs A100. What could cause a bottleneck this extreme?
The 170× slowdown of a point-tracking model on an NVIDIA T4 compared to an A100 suggests potential inefficiencies in 4D correlation volume processing and transformer layer execution, which may not scale well across different architectures.
Mathematics of Data Science
This book delves into the mathematical foundations of data science, covering essential topics such as Singular Value Decomposition, Linear Regression, and Deep Learning. Link to article
Fleet: Hierarchical Task-Based Abstraction for Megakernels on Multi-Die GPUs
Fleet introduces a multi-level task model that enhances GPU performance by mapping computation to memory scopes, specifically addressing the limitations of current programming models like CUDA/HIP in chiplet-based designs.
The qlora 2e-4 default is wrong under 10k samples and nobody talks about it
The default learning rate of 2e-4 for qlora is misleading for datasets under 10k samples, leading to overfitting and stagnant evaluation loss, as evidenced by personal experimentation.
Triton Plugin Extensions: Enabling TLX and Custom Compiler Passes Out of the Box
The Triton Plugin Extensions system in PyTorch-Triton 3.7 allows for dynamic loading of custom compiler passes and dialects at runtime, enabling Meta’s TLX for enhanced performance without the need for forking or recompiling.
ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level
ExTernD introduces a novel approach to ternary post-training quantization (PTQ) by decomposing matrices into two ternary matrices and an inner diagonal scaling matrix, allowing for arbitrary accuracy adjustments without fixed matrix sizes.