# Feb 12, 2025

## Daily

### Thomson Reuters wins first major AI copyright case in the US

**Thomson Reuters** has achieved a landmark victory in the first major **AI copyright case** in the US, reinforcing the notion that generative AI companies may face significant legal challenges regarding the use of copyrighted materials.

### Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

The study introduces a **novel language model architecture** that enhances test-time computation by leveraging **latent reasoning** through a recurrent depth approach, allowing for **arbitrary depth unrolling** without the need for specialized training data. [Link to article](https://arxiv.org/abs/2502.05171)

### DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

**DeepScaleR-1.5B-Preview** achieves **43.1% Pass@1 accuracy** on AIME2024, outperforming OpenAI’s o1-preview with a **1.5B model** trained on **40K high-quality math problems** over **3,800 A100 hours** ($4,500).

### A Year of Telepathy

**Neuralink's Telepathy** has enabled three individuals with paralysis to control digital devices using their thoughts, demonstrating significant advancements in brain-computer interface (BCI) technology over the past year.

### Fine-tuning is making big money—how?

**Fine-tuning** is a significant revenue driver for companies like Together AI, which has achieved **$100M+ ARR**, indicating a strong market demand for tailored AI solutions.

### League of Legends data scraping the hard and tedious way for fun

**High-fidelity datasets** for League of Legends can be created by reverse engineering the game engine, allowing for precise capture of gameplay events like ability usage and player movements, which are typically inaccessible through public APIs.

### Recurrent Latent Reasoning: Scaling Test-Time Compute in Language Models Without Token Generation

The paper introduces **"recurrent depth,"** allowing information to cycle through components multiple times, which enables **dynamic adjustment** of model depth during inference, enhancing efficiency.

### My experiments with Knowledge Distillation

**Knowledge Distillation** outperforms fine-tuning, especially when there is a significant size disparity between models, as evidenced by the performance metrics across various datasets.

### Automated Capability Discovery via Foundation Model Self-Exploration

**Automated Capability Discovery (ACD)** leverages a foundation model as a "scientist" to autonomously propose tasks that probe the abilities of other models, revealing thousands of capabilities and failures that would be difficult to uncover manually.

### Tracing mHuBERT model into a jit

The **mHuBERT model** has been successfully traced into a **JIT** format, facilitating the extraction of discrete **"semantic" tokens** from speech, which enhances usability in various applications.

### Novel Clustering Metric - The Jaccard-Concentration Index

The **Jaccard-Concentration Index (JCI)** is a novel Python library that evaluates clustering quality by combining the **Jaccard index** with a custom concentration score, offering a more nuanced view of cluster purity. [Learn more about JCI](https://pypi.org/project/jaccard-concentration-index/).

### TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models

**TAID** introduces a **novel knowledge distillation method** that utilizes a time-dependent intermediate distribution to effectively bridge the gap between student and teacher models, addressing challenges like capacity differences and mode collapse.

### LLMs Can Teach Themselves to Better Predict the Future

**Outcome-driven fine-tuning** enhances large language models' (LLMs) forecasting abilities by utilizing **self-play** to generate diverse reasoning trajectories, improving prediction accuracy by **7-10%** over base models.

### LLMs as Few-Shot Data Annotators for Multilingual Text Detoxification

**LLMs can generate high-quality parallel datasets** for text detoxification through few-shot learning, effectively creating toxic/non-toxic text pairs that retain semantic meaning while significantly reducing toxicity.

### The Continued Relevance of MaskNet: Leveraging Multiplicative Feature Interactions for CTR Prediction

**MaskNet's** innovative **instance-guided masking** technique enhances Click-Through Rate (CTR) prediction by dynamically emphasizing key features, proving its lasting relevance in a rapidly evolving AI landscape.