Jan 16, 2025

Daily

Titans: Learning to Memorize at Test Time

Titans introduce a neural long-term memory module that enhances attention mechanisms by allowing models to utilize both current and historical context, improving dependency modeling without the quadratic cost of traditional attention.

MatterGen: A new paradigm of materials design with generative AI

MatterGen revolutionizes materials design by utilizing generative AI to directly create novel materials based on specified properties, rather than relying on traditional screening methods that exhaust known candidates.

Homomorphic encryption in iOS 18

Homomorphic encryption enables Apple to perform operations on encrypted data, allowing users to search for images without exposing sensitive information, thus maintaining privacy while enhancing functionality.

100x defect tolerance: How we solved the yield problem

Cerebras has achieved a remarkable 100x defect tolerance in its Wafer Scale Engine 3 by designing significantly smaller cores, allowing for greater fault tolerance and yielding comparable results to traditional chips despite its massive size.

How I found & fixed 4 bugs in Microsoft's Phi-4 model

Microsoft's Phi-4 model, a 14B open-source competitor to GPT-4-o-mini, had four critical bugs affecting output quality, which were identified and fixed by the author, enhancing its performance significantly.

Transformer²: Self-Adaptive LLMs

Transformer² introduces a self-adaptive framework for large language models (LLMs) that enables real-time adaptation to unseen tasks by selectively adjusting weight matrix components, enhancing efficiency and performance.

Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget

SonyResearch's micro_diffusion demonstrates training a 1.16 billion parameter sparse transformer on a mere $1,890, utilizing 37M images to achieve an FID of 12.7 in zero-shot generation on the COCO dataset, showcasing cost-effective model training.

Train faster static embedding models with sentence transformers

Static embedding models can be trained 100x to 400x faster on CPU than traditional models while maintaining over 85% performance on benchmarks, enabling efficient on-device and edge computing applications.

Kaggle dataset: one of the input features has a >0.99 correlation with the target, yet most/all notebooks (20+) do not care?

One feature exhibits a >0.99 correlation with the target, yet most models fail to leverage this insight, leading to misleadingly high accuracy without proper normalization of the data.

Nvidia might do for desktop AI what it did for desktop gaming

NVIDIA's Project Digits aims to revolutionize desktop AI by enabling local processing of 200 billion-parameter models for around $3,000, making advanced AI accessible to a broader audience, including data scientists and researchers.

Imagine while Reasoning in Space: Multimodal Visualization-of-thought

Multimodal Visualization-of-Thought (MVoT) enhances reasoning in Multimodal Large Language Models (MLLMs) by integrating visual thinking, allowing models to generate image visualizations of their reasoning processes, thus bridging the gap between language and imagery.

MathReader: A Text-to-Speech System for Mathematical Documents Using OCR and Fine-tuned T5

MathReader innovatively combines OCR and a fine-tuned T5 model to convert mathematical documents into natural speech, treating mathematical notation as a unique language requiring specialized translation.

Researchers use AI to design proteins that block snake venom toxins

AI-driven protein design has successfully created inhibitors for snake venom toxins, showcasing the potential of computational tools in addressing complex biological challenges.

Multimodal Visualization-of-Thought: Enhancing MLLM Reasoning Through Visual Thinking

The Multimodal Visualization-of-Thought (MVoT) system enhances reasoning in large language models by integrating image generation, allowing AI to "visually think" and improve problem-solving capabilities.