# Jan 16, 2025

## Daily

### Titans: Learning to Memorize at Test Time

**Titans** introduce a **neural long-term memory module** that enhances attention mechanisms by allowing models to utilize both current and historical context, improving dependency modeling without the quadratic cost of traditional attention.

### MatterGen: A new paradigm of materials design with generative AI

**MatterGen** revolutionizes materials design by utilizing generative AI to directly create novel materials based on specified properties, rather than relying on traditional screening methods that exhaust known candidates.

### Homomorphic encryption in iOS 18

**Homomorphic encryption** enables Apple to perform operations on encrypted data, allowing users to search for images without exposing sensitive information, thus maintaining privacy while enhancing functionality.

### 100x defect tolerance: How we solved the yield problem

**Cerebras has achieved a remarkable 100x defect tolerance** in its Wafer Scale Engine 3 by designing significantly smaller cores, allowing for greater fault tolerance and yielding comparable results to traditional chips despite its massive size.

### How I found & fixed 4 bugs in Microsoft's Phi-4 model

**Microsoft's Phi-4 model**, a 14B open-source competitor to GPT-4-o-mini, had **four critical bugs** affecting output quality, which were identified and fixed by the author, enhancing its performance significantly.

### Transformer²: Self-Adaptive LLMs

**Transformer²** introduces a **self-adaptive framework** for large language models (LLMs) that enables real-time adaptation to unseen tasks by selectively adjusting weight matrix components, enhancing efficiency and performance.

### Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget

**SonyResearch's micro_diffusion** demonstrates training a **1.16 billion parameter sparse transformer** on a mere **$1,890**, utilizing **37M images** to achieve an **FID of 12.7** in zero-shot generation on the COCO dataset, showcasing cost-effective model training.

### Train faster static embedding models with sentence transformers

**Static embedding models can be trained 100x to 400x faster on CPU** than traditional models while maintaining over **85% performance** on benchmarks, enabling efficient on-device and edge computing applications.

### Kaggle dataset: one of the input features has a >0.99 correlation with the target, yet most/all notebooks (20+) do not care?

**One feature exhibits a >0.99 correlation with the target**, yet most models fail to leverage this insight, leading to misleadingly high accuracy without proper normalization of the data.

### Nvidia might do for desktop AI what it did for desktop gaming

**NVIDIA's Project Digits** aims to revolutionize desktop AI by enabling local processing of **200 billion-parameter models** for around **$3,000**, making advanced AI accessible to a broader audience, including data scientists and researchers.

### Imagine while Reasoning in Space: Multimodal Visualization-of-thought

**Multimodal Visualization-of-Thought (MVoT)** enhances reasoning in **Multimodal Large Language Models (MLLMs)** by integrating visual thinking, allowing models to generate image visualizations of their reasoning processes, thus bridging the gap between language and imagery.

### MathReader: A Text-to-Speech System for Mathematical Documents Using OCR and Fine-tuned T5

**MathReader** innovatively combines **OCR** and a **fine-tuned T5** model to convert mathematical documents into natural speech, treating mathematical notation as a unique language requiring specialized translation.

### Researchers use AI to design proteins that block snake venom toxins

**AI-driven protein design** has successfully created inhibitors for snake venom toxins, showcasing the potential of computational tools in addressing complex biological challenges.

### Multimodal Visualization-of-Thought: Enhancing MLLM Reasoning Through Visual Thinking

The **Multimodal Visualization-of-Thought (MVoT)** system enhances reasoning in large language models by integrating **image generation**, allowing AI to "visually think" and improve problem-solving capabilities.