Nov 27, 2025

Daily

• Got burned by an Apple ICLR paper — it was withdrawn after my Public Comment.

• A critical bug in an Apple ICLR paper's code led to unexpected low scores for a model, revealing potential 30% ground truth error rate in the dataset used, which was likely generated with poor quality control.

• Sutskever and LeCun assert that current large language models (LLMs) are reaching their limits, signaling a shift from an era of scaling to one focused on innovative research and new architectures, such as LeCun's JEPA.

• DRAM prices have surged by 171.8% in Q3 2025, driven largely by AI data center demand, with mainstream DDR5 modules now costing twice as much as earlier this year.

• Google's TPU, designed for AI inference, emerged from a need to efficiently handle massive data loads, outperforming traditional CPUs and GPUs by focusing on specific tasks like running TensorFlow neural networks.

• DeepSeek-Math-V2 is a significant advancement in mathematical modeling, enhancing the capabilities of AI in solving complex mathematical problems, as detailed in the DeepSeekMath_V2.pdf.

• Fara-7B is a 7 billion parameter agentic model that excels in automating computer tasks by interacting directly with user interfaces, achieving state-of-the-art performance with only ~16 steps per task compared to ~41 for similar models.

• GPL's propagation to AI models trained on GPL code raises significant legal questions about ownership and compliance, as the implications of using such code in AI development are still being debated.

• ICLR 2026 plans to crack down on LLM-generated submissions, with over 10% of papers and 20% of reviews identified as AI-generated, prompting strict policies for authors and reviewers.

• Training a filesystem using language models reveals that a minimal, effective filesystem can be created with surprisingly few operations, leveraging models like Qwen3-4b to achieve ~98% accuracy after fine-tuning on a dataset of 15,000 samples.

• The inverse hyperbolic sine activation function demonstrates superior performance in regression tasks, outperforming traditional functions like ReLU and sigmoid in cross-validation metrics such as WMAPE.

• The TSU emulator demonstrates a 1.3× speed advantage over MC Dropout in probabilistic machine learning tasks, with hardware projections indicating a 182× speedup for Bayesian neural networks, validated by significant hypothesis tests (p < 0.001) and large effect sizes (Cohen's d > 0.8).

• Current VRAM options for $15K per rack are limited, with some suggesting that a Mac Studio may outperform traditional high-end servers from NVIDIA, AMD, and Intel, challenging the notion of value in the market.

• Qwen3-VL is the most advanced vision-language model in its series, achieving superior performance across multimodal benchmarks with a 256K token capacity for text, images, and video integration.

• Subjective Depth Transformers (SDT) and Subjective Timescale Transformers (STT) utilize Bayesian surprise signals to dynamically optimize computation in decoder-only Transformers, enhancing efficiency for large-scale models and long sequences.

• Diffusion Language Models (DLMs) outperform autoregressive models by enabling faster inference through parallel decoding, yet traditional methods face an information-theoretic bottleneck that hampers efficiency.