ML Times

Feb 15, 2025

Daily

Microsoft study finds relying on AI kills critical thinking skills

• Over-reliance on AI tools leads to a significant decline in critical thinking skills, as users become less engaged in evaluating AI outputs and more dependent on them for task completion.

Benchmarking vision-language models on OCR in dynamic video environments

• This paper presents an open-source benchmark for evaluating Vision-Language Models (VLMs) on Optical Character Recognition (OCR) tasks in dynamic video environments, featuring a dataset of 1,477 annotated frames across various domains.

Did Semgrep Just Get a Lot More Interesting?

• Semgrep's integration with LLMs could revolutionize code generation and vulnerability detection, enabling automated rule creation and testing through a closed-loop system that learns from execution errors.

Oumi: An Open Source AI Lab

• Oumi is an open-source AI lab founded by ex-Google and Apple engineers, aiming to foster collaboration in AI development by making code, weights, and training data accessible to the community.

What's the Most Promising Successor to the Transformer?

• MAMBA emerges as a leading contender for Transformer successors, boasting linear inference efficiency, yet remains untested on large-scale models.

Show HN: Letting LLMs Run a Debugger

• LLM Debugger is a VSCode extension that enhances debugging by integrating real-time runtime context with large language models, allowing for faster and more accurate bug diagnosis.

Apache Iceberg now supports geospatial data types natively

• Apache Iceberg and Parquet now support GEO data types, enabling seamless integration of geospatial data into modern data ecosystems, which were previously disconnected from these advancements.

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

• The study introduces a novel language model architecture that enhances test-time computation by leveraging latent reasoning through a recurrent block, allowing for arbitrary depth unrolling without the need for specialized training data.

Goku Flow Based Video Generative Foundation Models

• Goku introduces a new family of joint image-and-video generation models utilizing rectified flow Transformers, achieving industry-grade performance through advanced techniques in data curation and model design.

In a showdown of psychotherapists vs. ChatGPT, the latter wins, new study finds

• A new study reveals that ChatGPT outperformed human psychotherapists in providing effective therapeutic responses, highlighting the potential of AI in mental health support.

First hybrid quantum supercomputer Reimei activated in Japan

• Japan's Reimei, the world's first hybrid quantum supercomputer, integrates 20 trapped-ion qubits into the Fugaku supercomputer, enhancing computational capabilities for complex problems in physics and chemistry.

Gary Marcus discusses AI's technical problems

• Gary Marcus argues that Large Language Models (LLMs) have hit a “point of diminishing returns,” indicating that merely increasing data will not resolve fundamental technical issues. This skepticism stems from his extensive research on neural networks, highlighting their limitations in extrapolation and understanding abstract concepts.

Can We Trust AI Benchmarks? A Review of Current Issues in AI Evaluation

• AI benchmarks are critical for assessing performance, capability, and safety of AI systems, yet they face scrutiny over their effectiveness in evaluating sensitive topics like high-impact capabilities and systemic risks.

Advancements in embedding-based retrieval at Pinterest Homefeed

• Pinterest's embedding-based retrieval has evolved with advanced feature crossing and ID embeddings, enhancing user engagement metrics by up to 1.2% through innovative model architectures like MaskNet and DHEN.

Evaluating Physical Concept Understanding in LLMs Through Abstract Grid-Based Tasks

• This study presents a structured assessment framework for evaluating physics understanding in LLMs, utilizing a test suite that spans mechanics, thermodynamics, and electromagnetism, with both quantitative and qualitative questions.