# Jan 17, 2025

## Daily

### Titans: Learning to Memorize at Test Time
- **Titans** introduce a **neural long-term memory module** that enhances attention mechanisms by allowing models to utilize both current and historical context, improving dependency modeling without the quadratic cost of traditional attention.

### MatterGen: A new paradigm of materials design with generative AI
- **MatterGen** revolutionizes materials design by utilizing generative AI to directly create novel materials based on specified properties, rather than relying on traditional screening methods that exhaust known candidates.

### Diffusion training from scratch on a micro-budget
- **SonyResearch's micro_diffusion** demonstrates training a **1.16 billion parameter sparse transformer** on a mere **$1,890**, utilizing **37M images** to achieve an **FID of 12.7** in zero-shot generation on the COCO dataset, showcasing cost-effective model training.

### Framework for Artificial Intelligence Diffusion
- The **Bureau of Industry and Security (BIS)** has introduced a framework to regulate the **export and diffusion of advanced AI technologies**, particularly focusing on AI model weights and advanced computing integrated circuits (ICs) to safeguard U.S. national security and foreign policy interests.

### Let's talk about AI and end-to-end encryption
- **AI's integration with end-to-end encryption raises significant privacy concerns**, as the need for powerful processing may lead to increased reliance on remote servers, potentially exposing sensitive data to unauthorized access.

### Grokking at the Edge of Numerical Stability [Research]
- **Grokking** reveals a phenomenon where models suddenly generalize after prolonged overfitting, yet the role of **regularization** in this process remains poorly understood, leading to the introduction of **Softmax Collapse (SC)** as a critical factor hindering grokking.

### MuJoco Playground
- **MuJoCo Playground** is an **open-source framework** designed for robot learning, enabling rapid training on diverse robotic platforms with a simple installation process via `pip install playground`.

### Dr. TVAM – Inverse Rendering for Tomographic Volumetric Additive Manufacturing
- **Dr.TVAM** is an **inverse rendering framework** designed for **tomographic volumetric additive manufacturing**, leveraging the Mitsuba renderer to optimize printing patterns through **physically-based differentiable rendering**.

### Skyvern Browser Agent 2.0: How We Reached State of the Art in Evals
- **Skyvern 2.0 achieves a remarkable score of 85.85% on the WebVoyager Eval**, showcasing its advanced capabilities in executing complex web navigation tasks autonomously, outperforming many closed-source competitors like Google Mariner.

### [R] Multimodal Visualization-of-Thought: Enhancing MLLM Reasoning Through Visual Thinking
- The **Multimodal Visualization-of-Thought (MVoT)** system enhances reasoning in large language models by integrating **image generation**, allowing AI to "visually think" and improve problem-solving capabilities.

### NVIDIA Releases NIM Microservices to Safeguard Applications for Agentic AI
- **NVIDIA's new NIM microservices** enhance the **safety, precision, and scalability** of generative AI applications, addressing critical enterprise concerns like trust and compliance through the **NVIDIA NeMo Guardrails** framework.

### Ideas: AI for materials discovery with Tian Xie and Ziheng Lu
- **AI tools MatterGen and MatterSim revolutionize materials discovery** by enabling the generation and simulation of materials tailored to specific properties, significantly enhancing efficiency in identifying novel materials for applications like batteries and energy storage.

### Research Focus: Week of January 13, 2025
- **Microsoft Research** introduces **privacy enhancements** for multiparty deep learning, significantly improving communication efficiency and speed in model training through **Differentially Private, secure Multiparty Computation (DP-MPC)** protocols, achieving up to **794 times** more efficiency than previous methods.

### Rational Tuning of LLM Cascades via Probabilistic Modeling
- This paper introduces a **probabilistic model** for the joint performance distribution of **LLM cascades**, enabling **rational tuning** of confidence thresholds through continuous optimization, which enhances performance predictability.

### To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation
- **Dynamic retrieval** in Retrieval-Augmented Generation can significantly enhance efficiency by only invoking external knowledge when the LLM lacks necessary information, addressing limitations in long-form question answering.