# Nov 9, 2024

## 1. SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup  
- **SVDQuant** introduces a **post-training quantization** method that reduces weights and activations to **4 bits**, achieving **3.6× memory reduction** and **8.7× speedup** on a **16GB 4090 GPU**.

## 2. LoRA vs. Full Fine-Tuning: An Illusion of Equivalence  
- **LoRA and full fine-tuning yield distinct weight matrix structures**, revealing that LoRA introduces new, high-ranking singular vectors termed **_intruder dimensions_**, which do not emerge in full fine-tuning.

## 3. [R] Most Time Series Anomaly Detection results are meaningless  
- **Much of the published Time Series Anomaly Detection (TSAD) research is deemed meaningless due to the overwhelming uncertainty in ground truth labels, which undermines the validity of algorithm comparisons.**

## 4. OpenCoder: Open-Source LLM for Coding  
- **OpenCoder** is a groundbreaking **open-access code LLM** that matches the performance of proprietary models while providing a comprehensive framework for reproducible scientific research, including model weights, training data, and detailed protocols.

## 5. Iterative α-(de)blending and Stochastic Interpolants  
- **Iterative α-(de)blending** simplifies diffusion models by using blending and deblending operations to map between probability distributions, yet it lacks a strong theoretical foundation, which is addressed by exploring **Stochastic Interpolants**.

## 6. Neural Optical Flow for PIV in Fluids  
- **Neural Optical Flow (NOF)** enhances **accuracy** and **robustness** in particle image velocimetry (PIV) by utilizing a **continuous neural-implicit representation** of the velocity field, which allows for efficient data assimilation and consistent regularization across stereo views.

## 7. FrontierMath: A benchmark for evaluating advanced mathematical reasoning in AI  
- **FrontierMath** introduces a benchmark of **hundreds of expert-level mathematics problems**, designed to assess AI's advanced reasoning capabilities, revealing that leading AI models solve less than **2%** of these problems compared to near-perfect scores on traditional benchmarks.

## 8. [R] Benchmarking Large Language Models with Integer Sequence Generation Tasks  
- This benchmark evaluates **large language models (LLMs)** on their ability to generate code for integer sequences from the **Online Encyclopedia of Integer Sequences (OEIS)**, revealing that the **o1 series** models excel in both accuracy and cheating detection compared to competitors like OpenAI and Google.

## 9. [D] Directions on drug-target interaction prediction  
- **Drug-target interaction (DTI) prediction** typically involves generating **target embeddings** with **PLMs** like **ESM2** and **drug embeddings** using **CLMs** such as **ChemBERTa**, followed by employing **cross-modal attention mechanisms** for integration.