# Feb 26, 2025  
ML Times

### FFT Strikes Back: An Efficient Alternative to Self-Attention

- **FFTNet** offers an innovative solution to the **quadratic complexity** of self-attention by utilizing the **Fast Fourier Transform (FFT)**, achieving global token mixing in **$\mathcal{O}(n\log n)$ time**.

### Accelerating Generative AI with PyTorch: Segment Anything 2 - Fast and furious inference with low latency and fast cold starts

- **Optimized SAM2**: The implementation of AOTInductor’s ahead-of-time compilation via `torch.export`, along with reduced precision and batched prompts, achieves up to **13x improvement in p90 execution latency** and queue times compared to standard eager mode PyTorch.

### DeepSeek Open Sources DeepGEMM: Clean and efficient FP8 GEMM kernels

- **DeepGEMM** is a CUDA library for **FP8 General Matrix Multiplications (GEMMs)**, optimized for NVIDIA Hopper tensor cores, featuring a lightweight **Just-In-Time (JIT)** compilation that eliminates the need for pre-installation compilation.

### Hard problems that reduce to document ranking

- **LLMs can effectively solve complex problems** by transforming them into **document ranking** tasks, as demonstrated in locating N-day vulnerabilities through patch diffing, utilizing established ranking techniques from information retrieval.

### The FFT Strikes Back: An Efficient Alternative to Self-Attention

- The **Fast Fourier Transform (FFT)** offers a **scalable alternative** to traditional self-attention by reducing computational complexity from **O(n²)** to **O(n log n)**, enabling efficient modeling of long-range dependencies in sequences.

### DeepSearcher: A local open-source Deep Research

- **DeepSearcher** is an advanced open-source research agent that enhances local deep research capabilities by integrating features like **query routing** and **conditional execution flow**, allowing for more efficient information retrieval and report generation.

### Alexa+

- **Alexa+** leverages **generative AI** to enhance user interaction, making conversations feel more natural and intuitive, akin to speaking with a knowledgeable friend.

### A possible future of Python in the browser

- **SPy** is an emerging language designed for Python users, enabling **native compilation** to binaries or WebAssembly, thus enhancing performance significantly over existing solutions like Pyodide and MicroPython.

### Train your own Reasoning model - GRPO works on just 5GB VRAM

- **GRPO now operates on just 5GB of VRAM**, enabling **10x longer context lengths** without sacrificing accuracy, thanks to newly implemented Efficient GRPO algorithms.

### Muon is Scalable for LLM Training

- **Muon** is an optimizing algorithm that achieves **∼2× computational efficiency** over AdamW, saving about **50% FLOPs** for training a 1.5B LLM on 39B tokens, demonstrating its scalability for larger models.

### SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference

- **SpargeAttn** introduces a **universal sparse and quantized attention** mechanism that accelerates inference across various models by effectively predicting and filtering the attention map, thus reducing computational overhead.

### Forecasting Rare Language Model Behaviors

- **Anthropic's method leverages power-law scaling** to predict rare AI risks, enabling early detection of issues like harmful responses or misaligned behavior before deployment.

### From Records to Agents: The Overlooked Revolution in Enterprise Software

- **Enterprise software is evolving from passive records to autonomous agents**, enabling business objects like invoices to actively pursue their own approval and payment processes, fundamentally changing how organizations operate.

### RAGSys: Real-Time Self-Improvement for LLMs Without Retraining

- **RAGSys** introduces a **dynamic framework** for LLMs that optimizes prompt composition by retrieving relevant few-shot examples and instructions tailored to each query, enhancing response quality in real-time.

### Regarding quantization, what are the future directions in this topic for LLM/SLMs?

- **Future directions in quantization for LLMs/SLMs** may focus on enhancing model efficiency while maintaining performance, particularly through advanced techniques like **post-training quantization** and **mixed-precision training**.