1. SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup
SVDQuant introduces a post-training quantization method that reduces weights and activations to 4 bits, achieving 3.6× memory reduction and 8.7× speedup on a 16GB 4090 GPU.
2. LoRA vs. Full Fine-Tuning: An Illusion of Equivalence
LoRA and full fine-tuning yield distinct weight matrix structures, revealing that LoRA introduces new, high-ranking singular vectors termed intruder dimensions, which do not emerge in full fine-tuning.
3. [R] Most Time Series Anomaly Detection results are meaningless
Much of the published Time Series Anomaly Detection (TSAD) research is deemed meaningless due to the overwhelming uncertainty in ground truth labels, which undermines the validity of algorithm comparisons.
4. OpenCoder: Open-Source LLM for Coding
OpenCoder is a groundbreaking open-access code LLM that matches the performance of proprietary models while providing a comprehensive framework for reproducible scientific research, including model weights, training data, and detailed protocols.
5. Iterative α-(de)blending and Stochastic Interpolants
Iterative α-(de)blending simplifies diffusion models by using blending and deblending operations to map between probability distributions, yet it lacks a strong theoretical foundation, which is addressed by exploring Stochastic Interpolants.
6. Neural Optical Flow for PIV in Fluids
Neural Optical Flow (NOF) enhances accuracy and robustness in particle image velocimetry (PIV) by utilizing a continuous neural-implicit representation of the velocity field, which allows for efficient data assimilation and consistent regularization across stereo views.
7. FrontierMath: A benchmark for evaluating advanced mathematical reasoning in AI
FrontierMath introduces a benchmark of hundreds of expert-level mathematics problems, designed to assess AI's advanced reasoning capabilities, revealing that leading AI models solve less than 2% of these problems compared to near-perfect scores on traditional benchmarks.
8. [R] Benchmarking Large Language Models with Integer Sequence Generation Tasks
This benchmark evaluates large language models (LLMs) on their ability to generate code for integer sequences from the Online Encyclopedia of Integer Sequences (OEIS), revealing that the o1 series models excel in both accuracy and cheating detection compared to competitors like OpenAI and Google.
9. [D] Directions on drug-target interaction prediction
Drug-target interaction (DTI) prediction typically involves generating target embeddings with PLMs like ESM2 and drug embeddings using CLMs such as ChemBERTa, followed by employing cross-modal attention mechanisms for integration.