# Feb 27, 2025

## Articles

- **FFT Strikes Back: An Efficient Alternative to Self-Attention**  
  **FFTNet** offers an innovative solution to the **quadratic complexity** of self-attention by utilizing the **Fast Fourier Transform (FFT)**, achieving global token mixing in **$\mathcal{O}(n\log n)$ time**.

- **DualPipe: An innovative bidirectional pipeline parallelism algorithm**  
  **DualPipe** is a **bidirectional pipeline parallelism algorithm** that maximizes the overlap of forward and backward computation-communication phases, effectively minimizing pipeline bubbles and enhancing efficiency.

- **Accelerating Generative AI with PyTorch: Segment Anything 2 - Fast and furious inference with low latency and fast cold starts**  
  **Optimized SAM2**: The implementation of AOTInductor’s ahead-of-time compilation via `torch.export`, along with reduced precision and batched prompts, achieves up to **13x improvement in p90 execution latency** and queue times compared to standard eager mode PyTorch.

- **The FFT Strikes Back: An Efficient Alternative to Self-Attention**  
  The **Fast Fourier Transform (FFT)** offers a **scalable alternative** to traditional self-attention by reducing computational complexity from **O(n²)** to **O(n log n)**, enabling efficient modeling of long-range dependencies in sequences.

- **Train your own Reasoning model - GRPO works on just 5GB VRAM**  
  **GRPO now operates on just 5GB of VRAM**, enabling **10x longer context lengths** without sacrificing accuracy, thanks to newly implemented Efficient GRPO algorithms.

- **RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning (2023)**  
  **RoboPianist** employs **deep reinforcement learning** to train anthropomorphic robot hands for **dexterous piano playing**, releasing a benchmark dataset to enhance high-dimensional control research.

- **Beyond Dot Products: Retrieval with Learned Similarities**  
  **New approach**: The paper introduces **Mixture of Logits (MoL)**, a method that enables **learned similarity functions**, surpassing traditional dot product methods in efficiency and effectiveness for **recommendation systems** and **question answering**.

- **Flawed Diamonds Make Perfect Quantum Sensors**  
  **Flawed diamonds** possess unique properties that enhance their effectiveness as **quantum sensors**, enabling applications in diverse fields such as **chip technology** and **art history**.

- **RAGSys: Real-Time Self-Improvement for LLMs Without Retraining**  
  **RAGSys** introduces a **dynamic framework** for LLMs that optimizes prompt composition by retrieving relevant few-shot examples and instructions tailored to each query, enhancing response quality in real-time.

- **Forecasting Rare Language Model Behaviors**  
  **Anthropic's method leverages power-law scaling** to predict rare AI risks, enabling early detection of issues like harmful responses or misaligned behavior before deployment.

- **JOSH: Self-Improving LLMs for Tool Use Without Human Feedback**  
  **JOSH** (Juxtaposed Outcomes for Simulation Harvesting) is a **self-alignment algorithm** that enhances LLMs' tool-using abilities autonomously, achieving a **74% improvement** in success rates on the ToolWOZ benchmark for Llama3-8B models.

- **FFTNet: Linear-Time Global Token Mixing via Adaptive Spectral Filtering**  
  **FFTNet** replaces **quadratic complexity** self-attention with **linear complexity** using Fast Fourier Transforms, enabling efficient global token mixing while preserving performance.

- **Sugaku: AI tools for exploratory math research, based on training on a database of millions of paper examples**  
  **Sugaku** is an AI platform that enhances mathematical research by connecting users with relevant literature, generating ideas, and answering queries based on a vast database of over **7 million papers** and **4 million authors**.

- **Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization**  
  **Drop-Upcycling** enhances the **Mixture of Experts (MoE)** architecture by combining pre-trained dense model knowledge with selective weight re-initialization, leading to improved long-term performance.

- **Explore How RTX AI PCs and Workstations Supercharge AI Development at NVIDIA GTC 2025**  
  **RTX AI PCs** are revolutionizing AI development by leveraging **Tensor Cores** for enhanced performance in applications like digital humans and chatbots, showcased at [GTC 2025](https://www.nvidia.com/gtc/?nvid=nv-int-bnr-463583) from March 17–21.