# Nov 27, 2025

## Daily

- **Got burned by an Apple ICLR paper** — it was withdrawn after my Public Comment.

- A **critical bug** in an Apple ICLR paper's code led to unexpected low scores for a model, revealing potential **30% ground truth error rate** in the dataset used, which was likely generated with poor quality control.

- **Sutskever and LeCun assert that current large language models (LLMs) are reaching their limits**, signaling a shift from an era of scaling to one focused on innovative research and new architectures, such as LeCun's **JEPA**.

- **DRAM prices have surged** by **171.8%** in Q3 2025, driven largely by **AI data center demand**, with mainstream DDR5 modules now costing **twice** as much as earlier this year.

- **Google's TPU**, designed for AI inference, emerged from a need to efficiently handle massive data loads, outperforming traditional CPUs and GPUs by focusing on specific tasks like running TensorFlow neural networks.

- **DeepSeek-Math-V2** is a significant advancement in mathematical modeling, enhancing the capabilities of AI in solving complex mathematical problems, as detailed in the [DeepSeekMath_V2.pdf](https://github.com/deepseek-ai/DeepSeek-Math-V2/blob/main/DeepSeekMath_V2.pdf).

- **Fara-7B** is a **7 billion parameter** agentic model that excels in automating computer tasks by interacting directly with user interfaces, achieving **state-of-the-art performance** with only **~16 steps per task** compared to ~41 for similar models.

- **GPL's propagation to AI models** trained on GPL code raises significant legal questions about ownership and compliance, as the implications of using such code in AI development are still being debated.

- **ICLR 2026** plans to **crack down** on LLM-generated submissions, with over **10% of papers** and **20% of reviews** identified as AI-generated, prompting strict policies for authors and reviewers.

- **Training a filesystem using language models** reveals that a minimal, effective filesystem can be created with surprisingly few operations, leveraging models like Qwen3-4b to achieve **~98% accuracy** after fine-tuning on a dataset of 15,000 samples.

- The **inverse hyperbolic sine** activation function demonstrates superior performance in regression tasks, outperforming traditional functions like ReLU and sigmoid in cross-validation metrics such as **WMAPE**.

- The **TSU emulator** demonstrates a **1.3× speed advantage** over MC Dropout in probabilistic machine learning tasks, with hardware projections indicating a **182× speedup** for Bayesian neural networks, validated by significant hypothesis tests (p < 0.001) and large effect sizes (Cohen's d > 0.8).

- **Current VRAM options for $15K per rack** are limited, with some suggesting that a **Mac Studio** may outperform traditional high-end servers from NVIDIA, AMD, and Intel, challenging the notion of value in the market.

- **Qwen3-VL** is the most advanced vision-language model in its series, achieving **superior performance** across multimodal benchmarks with a **256K token** capacity for text, images, and video integration.

- **Subjective Depth Transformers (SDT)** and **Subjective Timescale Transformers (STT)** utilize **Bayesian surprise signals** to dynamically optimize computation in decoder-only Transformers, enhancing efficiency for large-scale models and long sequences.

- **Diffusion Language Models (DLMs)** outperform autoregressive models by enabling **faster inference** through parallel decoding, yet traditional methods face an **information-theoretic bottleneck** that hampers efficiency.