# Feb 11, 2025

### News Highlights

- **Undergraduate Andrew Krapivin** and colleagues have developed a new type of **hash table** that significantly speeds up data searches, contradicting a **40-year-old conjecture** about their efficiency.

- The study introduces a **novel language model architecture** that enhances test-time computation by leveraging **latent reasoning** through a recurrent depth approach, allowing for **arbitrary depth unrolling** without the need for specialized training data. [Link to article](https://arxiv.org/abs/2502.05171)

- A **2023 study** from UC Irvine reveals that CAPTCHAs serve as a **tracking cookie farm**, generating nearly **$1 trillion** for Google while wasting **819 million hours** of user time on tasks like identifying traffic lights.

- **DeepScaleR-1.5B-Preview** achieves **43.1% Pass@1 accuracy** on AIME2024, outperforming OpenAI’s o1-preview with a **1.5B model** trained on **40K high-quality math problems** over **3,800 A100 hours** ($4,500).

- **Outcome-driven fine-tuning** enhances LLM forecasting by utilizing self-play to create diverse reasoning trajectories, improving prediction accuracy without human input.

- **Thomson Reuters** has achieved a landmark victory in the first major **AI copyright case** in the US, reinforcing the notion that generative AI companies may face significant legal challenges regarding the use of copyrighted materials.

- **Fine-tuning** is a significant revenue driver for companies like Together AI, which has achieved **$100M+ ARR**, indicating a strong market demand for tailored AI solutions.

- **AI agents are transforming SaaS** by automating interactions, making traditional user interfaces obsolete as agents handle tasks directly, thus shifting the value proposition from user experience to backend functionality.

- **Knowledge Distillation** outperforms fine-tuning, especially when there is a significant size disparity between models, as evidenced by the performance metrics across various datasets.

- **Personalized text generation** using open-sourced Generative AI models enables the creation of tailored writing styles, raising significant **privacy and safety concerns** as malicious actors can impersonate individuals with minimal data.

- A group of authors is **suing AI vendors** for allegedly training models like Meta's LLaMA on copyrighted material without consent, raising significant legal questions about AI training practices.

- The paper introduces **"recurrent depth,"** allowing information to cycle through components multiple times, which enables **dynamic adjustment** of model depth during inference, enhancing efficiency.

- The **mHuBERT model** has been successfully traced into a **JIT** format, facilitating the extraction of discrete **"semantic" tokens** from speech, which enhances usability in various applications.

- **KL divergence** as a reward model in **RL** for LLMs could yield sequences with significantly lower KL divergence than those generated by the pretrained model, potentially enhancing **coherence** in outputs.

- **Pretraining significantly influences** the effectiveness of reinforcement learning (RL) in large language models (LLMs), suggesting that the interplay between pretraining size and RL compute budgets is crucial for optimizing model performance.