# Jul 16, 2026

## Inkling: Our Open-Weights Model

- **Inkling** is a **Mixture-of-Experts transformer** model with **975 billion total parameters** and **41 billion active**, designed for multimodal reasoning across text, images, and audio, and is available for customization on Tinker.

## Kimi K3: Open Frontier Intelligence

- **Kimi K3** is a groundbreaking **2.8 trillion-parameter model** that integrates advanced **Kimi Delta Attention** and **Attention Residuals**, enabling it to excel in long-horizon coding and knowledge work while being the first open model in the **3T-class** category.

## German AI consortium releases Soofi S, an open 30B model that tops benchmarks

- **Soofi S**, an open-source language model developed by a German consortium, utilizes a **hybrid architecture** that activates only **3.2 billion of its 31.6 billion parameters** per token, ensuring efficient processing even with lengthy inputs.

## Schema Harness Achieves ~99% on Arc‑AGI‑3 Public

- **Schema achieves ~99% RHAE on ARC-AGI-3 Public set**, demonstrating a significant leap in action efficiency by integrating state grounding and mechanism discovery into a single editable program, allowing for real-time model adjustments based on observations.

## Detecting LLM-Generated Texts with “Classical” Machine Learning

- **Mainstream LLM-generated text can be distinguished from human-written content with approximately 85% accuracy using traditional machine learning models like Linear SVC and Naive Bayes.** This detection leverages statistical patterns inherent in AI-generated text, which are often consistent and detectable compared to the variability of human writing.

## Mechanistic interpretability: a first paper on disentangling a convolutional neuron

- The study reveals that the **Hadamard product** of a neuron's receptive field and weights effectively represents what the neuron detects, allowing for the clustering of patterns like **cars, cats, and dogs** into clean monosemantic groups, alongside unexpected clusters such as letters and human faces.

## Designing APIs for Agents

- **Designing APIs for agents** requires a shift from human-centric principles, emphasizing **clarity and explicitness** to accommodate AI's ability to process extensive documentation and generate code rapidly.

## High-Bandwidth Flash offers efficient storage for model weights

- **High-Bandwidth Flash** technology enables **dramatically improved read speeds** by stacking flash memory dies, addressing the storage limitations faced by AI models.

## Pseudpocalypse

- **Pseudpocalypse** suggests that unique writing styles can be statistically linked to authors, potentially compromising pseudonymity as machine learning advances in text analysis.

## PyTorch model running 170x slower on T4 vs A100. What could cause a bottleneck this extreme?

- The **170× slowdown** of a point-tracking model on an NVIDIA T4 compared to an A100 suggests potential inefficiencies in **4D correlation volume** processing and transformer layer execution, which may not scale well across different architectures.

## Mathematics of Data Science

- This book delves into the **mathematical foundations** of data science, covering essential topics such as **Singular Value Decomposition**, **Linear Regression**, and **Deep Learning**. [Link to article](https://arxiv.org/abs/2607.11938)

## Fleet: Hierarchical Task-Based Abstraction for Megakernels on Multi-Die GPUs

- **Fleet** introduces a **multi-level task model** that enhances GPU performance by mapping computation to memory scopes, specifically addressing the limitations of current programming models like CUDA/HIP in chiplet-based designs.

## The qlora 2e-4 default is wrong under 10k samples and nobody talks about it

- The **default learning rate of 2e-4** for qlora is misleading for datasets under **10k samples**, leading to overfitting and stagnant evaluation loss, as evidenced by personal experimentation.

## Triton Plugin Extensions: Enabling TLX and Custom Compiler Passes Out of the Box

- The **Triton Plugin Extensions** system in PyTorch-Triton 3.7 allows for **dynamic loading** of custom compiler passes and dialects at runtime, enabling **Meta’s TLX** for enhanced performance without the need for forking or recompiling.

## ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level

- **ExTernD** introduces a novel approach to **ternary post-training quantization (PTQ)** by decomposing matrices into two ternary matrices and an inner diagonal scaling matrix, allowing for **arbitrary accuracy** adjustments without fixed matrix sizes.