Feb 12, 2025

Daily

Thomson Reuters wins first major AI copyright case in the US

Thomson Reuters has achieved a landmark victory in the first major AI copyright case in the US, reinforcing the notion that generative AI companies may face significant legal challenges regarding the use of copyrighted materials.

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

The study introduces a novel language model architecture that enhances test-time computation by leveraging latent reasoning through a recurrent depth approach, allowing for arbitrary depth unrolling without the need for specialized training data. Link to article

DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL

DeepScaleR-1.5B-Preview achieves 43.1% Pass@1 accuracy on AIME2024, outperforming OpenAI’s o1-preview with a 1.5B model trained on 40K high-quality math problems over 3,800 A100 hours ($4,500).

A Year of Telepathy

Neuralink's Telepathy has enabled three individuals with paralysis to control digital devices using their thoughts, demonstrating significant advancements in brain-computer interface (BCI) technology over the past year.

Fine-tuning is making big money—how?

Fine-tuning is a significant revenue driver for companies like Together AI, which has achieved $100M+ ARR, indicating a strong market demand for tailored AI solutions.

League of Legends data scraping the hard and tedious way for fun

High-fidelity datasets for League of Legends can be created by reverse engineering the game engine, allowing for precise capture of gameplay events like ability usage and player movements, which are typically inaccessible through public APIs.

Recurrent Latent Reasoning: Scaling Test-Time Compute in Language Models Without Token Generation

The paper introduces "recurrent depth," allowing information to cycle through components multiple times, which enables dynamic adjustment of model depth during inference, enhancing efficiency.

My experiments with Knowledge Distillation

Knowledge Distillation outperforms fine-tuning, especially when there is a significant size disparity between models, as evidenced by the performance metrics across various datasets.

Automated Capability Discovery via Foundation Model Self-Exploration

Automated Capability Discovery (ACD) leverages a foundation model as a "scientist" to autonomously propose tasks that probe the abilities of other models, revealing thousands of capabilities and failures that would be difficult to uncover manually.

Tracing mHuBERT model into a jit

The mHuBERT model has been successfully traced into a JIT format, facilitating the extraction of discrete "semantic" tokens from speech, which enhances usability in various applications.

Novel Clustering Metric - The Jaccard-Concentration Index

The Jaccard-Concentration Index (JCI) is a novel Python library that evaluates clustering quality by combining the Jaccard index with a custom concentration score, offering a more nuanced view of cluster purity. Learn more about JCI.

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models

TAID introduces a novel knowledge distillation method that utilizes a time-dependent intermediate distribution to effectively bridge the gap between student and teacher models, addressing challenges like capacity differences and mode collapse.

LLMs Can Teach Themselves to Better Predict the Future

Outcome-driven fine-tuning enhances large language models' (LLMs) forecasting abilities by utilizing self-play to generate diverse reasoning trajectories, improving prediction accuracy by 7-10% over base models.

LLMs as Few-Shot Data Annotators for Multilingual Text Detoxification

LLMs can generate high-quality parallel datasets for text detoxification through few-shot learning, effectively creating toxic/non-toxic text pairs that retain semantic meaning while significantly reducing toxicity.

The Continued Relevance of MaskNet: Leveraging Multiplicative Feature Interactions for CTR Prediction

MaskNet's innovative instance-guided masking technique enhances Click-Through Rate (CTR) prediction by dynamically emphasizing key features, proving its lasting relevance in a rapidly evolving AI landscape.