# Dec 17, 2024

## Daily

### Veo 2: Our video generation model
- **Veo 2** is a cutting-edge video generation model that produces **realistic motion** and **high-quality output** up to **4K resolution**, significantly enhancing detail and reducing artifacts compared to previous models.

### New LLM optimization technique slashes memory costs up to 75%
- **New technique** called **universal transformer memory** developed by Sakana AI enables LLMs to optimize memory usage, achieving up to **75% reduction** in cache memory costs while enhancing performance on long-context tasks.

### Konwinski Prize
- **$1M K Prize** will reward the first open source AI to achieve **90% accuracy** on a new, contamination-free version of **SWE-bench**, promoting integrity in AI benchmarking.

### In Search of a Faster SQLite
- Researchers at the University of Helsinki and Cambridge have achieved a **100x reduction in tail latency** for SQLite by implementing **asynchronous I/O** and **disaggregated storage** in their paper, [“Serverless Runtime / Database Co-Design With Asynchronous I/O”](https://penberg.org/papers/penberg-edgesys24.pdf).

### 🤗Welcome the Falcon 3 Family of Open Models!
- **Falcon 3** introduces a family of **decoder-only large language models** under 10 billion parameters, developed by the **Technology Innovation Institute**, enhancing performance in science, math, and coding through innovative training techniques.

### 
### [R] Scaling test-time compute with open models!
- **Hugging Face researchers** have successfully **outperformed Llama 70B** using Llama 3B on MATH by integrating **step-wise reward models** with **tree-search algorithms** for enhanced test-time compute efficiency.

### [R] SVGFusion: Scalable Text-to-SVG Generation via Vector Space Diffusion
- **SVGFusion** introduces a novel **Text-to-SVG model** that leverages a continuous latent space for vector graphics, enhancing the generation of scalable vector graphics from text without relying on traditional discrete language models.

### RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation
- **RetroLLM** integrates retrieval and generation into a single framework, allowing large language models to produce **fine-grained evidence** directly from the corpus, thus addressing hallucination issues effectively.

### [R] Optimizing LLM merging to reduce performance tradeoffs
- **Optimizing LLM merging** can yield **Pareto-optimal models** by effectively combining checkpoints trained under varying conditions, enhancing performance across multiple tasks without the need for extensive hyperparameter tuning.

### NVIDIA Unveils Its Most Affordable Generative AI Supercomputer
- **NVIDIA's Jetson Orin Nano Super** offers a **1.7x increase in generative AI performance** at a significantly reduced price of **$249**, making advanced AI capabilities accessible to hobbyists and developers alike.

### Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture
- **Combining sequence transformation and state transformation** enhances foundation model efficiency, with rotary position embedding reducing perplexity by over **4%** in hybrid quadratic causal self-attention and state space duality.

### [Discussion] [D] Introducing TLR: Training AI Simultaneously Across Three Environments with Shared Learning
- **TLR (Triple Layer Training)** is a **novel reinforcement learning framework** that enables a single agent to train across **three diverse environments**—Cart Pole, Lunar Lander, and Space Invader—simultaneously, enhancing learning through shared experiences.

### SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
- **SepLLM** introduces a novel approach to compress segments of data into **separator tokens**, significantly enhancing the **inference speed** of Large Language Models (LLMs) while preserving performance.

### [R] Continuous Numerical Tokenization for Scientific Language Models Improves Out-of-Distribution Performance
- The **xVal** method introduces **continuous numerical tokenization**, allowing language models to represent numbers as continuous values, which enhances their understanding of **numerical patterns** in scientific texts.

### FastVideo: a lightweight framework for accelerating large video diffusion models
- **FastVideo** is a **lightweight framework** designed to accelerate large video diffusion models, achieving up to **8x inference speedup** with models like **FastHunyuan** and **FastMochi**.