# Apr 30, 2025

## The Leaderboard Illusion

- **Chatbot Arena** has become a pivotal leaderboard for AI systems, yet it suffers from **systematic biases** due to undisclosed testing practices that favor select providers, allowing them to manipulate scores by choosing the best results from multiple tests. [Link to article](https://arxiv.org/abs/2504.20879)

## Chain of Recursive Thoughts: Make AI think harder by making it argue with itself

- **CoRT (Chain of Recursive Thoughts)** enhances AI by enabling it to **argue with itself**, leading to significantly improved responses, particularly in programming tasks with the Mistral 3.1 24B model.

## Xiaomi MiMo Reasoning Model

- **MiMo-7B** is a groundbreaking series of models designed specifically for reasoning tasks, demonstrating superior performance in mathematics and code reasoning, even outperforming larger models like the 32B variants.

## DeepSeek-Prover-V2

- **DeepSeek-Prover-V2** is an advanced open-source model for formal theorem proving in Lean 4, utilizing a recursive theorem proving pipeline to synthesize cold-start reasoning data from subgoal decomposition, enhancing its performance in mathematical reasoning tasks.

## Everything we announced at our first LlamaCon

- **LlamaCon** introduced the **Llama API**, a limited preview that merges the advantages of closed model APIs with the flexibility of open-source, enabling developers to create custom models effortlessly.

## Bamba: An open-source LLM that crosses a transformer with an SSM

- **Bamba**, IBM's new **open-source LLM**, merges the **speed of state-space models** with the **expressive power of transformers**, promising enhanced performance for long sequences.

## An illustrated guide to automatic sparse differentiation

- **Automatic Sparse Differentiation (ASD)** leverages the **sparsity** of Hessians and Jacobians to significantly enhance computational efficiency in machine learning applications, yet it remains underutilized compared to traditional automatic differentiation (AD).

## Awesome List on AI for Security

- **Curated resource**: The **Awesome AI for Security** repository compiles tools, papers, and datasets focused on leveraging **AI technologies** like **Large Language Models (LLMs)** for cybersecurity tasks, enhancing security operations through modern methodologies.

## [R] Bringing Emotions to Recommender Systems: A Deep Dive into Empathetic Conversational Recommendation

- **ECR (Empathetic Conversational Recommender)** enhances traditional systems by integrating **user emotions** into both item recommendations and dialogue generation, utilizing emotion-aware representations and feedback mechanisms.

## [P] hacking on graph-grounded retrieval for SEC filings + an AI “legal pen-tester”—looking for feedback & maybe collaborators

- **Graph-first ingestion & retrieval** aims to transform **300-page SEC filings** into **traceable embeddings** with a target of **50 ms query latency**, currently under a **patent-pending pipeline** development.

## [Discussion] I trained a 7B LLM with only 8GB of VRAM using symbolic compression MemoryCore benchmark results

- A **symbolic compression pipeline** enabled training a **7B parameter LLM** on just **8GB of VRAM**, utilizing techniques like **symbolic tokenization** and **modular encoding layers** to maintain model quality while reducing compute demands.

## DYNAMAX: Dynamic computing for Transformers and Mamba based architectures

- **DYNAMAX** introduces a novel framework that integrates **early exits (EEs)** into **Mamba architectures**, enhancing their role as efficient classifiers for both Mamba and transformer-based LLMs, thus addressing a gap in current research.

## Reinforcement Learning for LLM Reasoning Under Memory Constraints

- **Reinforcement learning (RL) techniques** enhance reasoning in large language models (LLMs) under **memory constraints**, introducing S-GRPO and T-SPMO for improved performance on benchmarks like SVAMP.

## Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges

- **LLMs can generate source code** but are susceptible to **hallucinations**, which are incorrect outputs that can be difficult to detect and rectify, especially in complex codebases.

## Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?

- **LLMs exhibit significant variability** in detecting intrinsic hallucinations during paraphrasing and machine translation, influenced by factors such as model size and prompt choice, as evaluated through the HalluciGen task.