# Aug 2, 2025

## Topics Covered

- **Deep Think** is now available in the **Gemini app** for Google AI Ultra subscribers, featuring advanced parallel thinking and novel reinforcement learning techniques that enhance problem-solving capabilities significantly.

- **Cerebras Code** introduces two subscription plans, **Cerebras Code Pro** ($50/month) and **Code Max** ($200/month), offering access to the **Qwen3-Coder** model, which generates code at **2,000 tokens per second** with a **131k-token context window**.

- **DeepSeek won the best paper award at ACL 2025**: **NSA** (Natively trainable Sparse Attention) introduces a **dynamic hierarchical sparse strategy** that enhances long-context modeling by balancing **coarse-grained token compression** with **fine-grained token selection** for improved efficiency and context awareness.

- **Native Sparse Attention (NSA)** introduces a **dynamic hierarchical sparse strategy** that enhances long-context modeling efficiency while maintaining model performance, integrating hardware-aligned optimizations for modern architectures.

- **Hiding secret codes in light protects against fake videos**: **Cornell researchers have developed a method to watermark light in videos**, enabling the detection of manipulated footage by embedding secret codes in lighting fluctuations that remain imperceptible to the human eye.

- **Deep agents** enhance the basic LLM architecture by integrating a **planning tool**, **sub agents**, **file system access**, and a **detailed prompt**, enabling them to tackle complex tasks over extended periods effectively.

- **Ferroelectric Helps Break Transistor Limits**: **Negative capacitance** in GaN transistors enhances performance by reducing **leakage current**, enabling devices to operate more efficiently without traditional trade-offs.

- The **ASI-Arch paper** reveals that an **automated AI search** identified **106 novel neural architectures**, many of which surpass traditional human-designed models, showcasing innovative combinations of techniques that challenge conventional design intuition. [Read the paper](https://news.lambda.ai/news/arxiv.org/abs/2507.18074).

- The **Iceberg spec** fails to effectively address the **metadata problem** in large Data Lakes, leading to excessive complexity and inefficiency in managing data, as it requires clients to handle extensive bookkeeping for simple changes.

- **GPU snapshotting** significantly enhances performance, cutting cold start times for large models by **up to 12x**, thanks to NVIDIA's new [CUDA checkpoint/restore API](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__CHECKPOINT.html).

- **Tri-70B-preview-SFT** is a **70 billion-parameter language model** trained on approximately **1.5 trillion tokens**, utilizing a **pure supervised fine-tuning (SFT)** approach without reinforcement learning from human feedback (RLHF), making it a valuable baseline for alignment research.

- **Weight tying in LLMs** appears to transform the last MLP layer into the **true unembedding**, effectively simplifying the model's architecture by reducing computational redundancy.

- The **GPT with Modified Memorizing Transformer** enhances traditional transformer models by integrating a memorization mechanism, allowing for improved retention of information over longer contexts.

- The **Model Synthesis Architecture (MSA)** leverages both **distributed and symbolic representations** to create tailored mental models for novel situations, enhancing reasoning capabilities through probabilistic programming.