Show HN: Llama 3.2 Interpretability with Sparse Autoencoders
Llama 3's interpretability is enhanced through Sparse Autoencoders (SAEs), which aim to separate superimposed neuron activations into distinct, interpretable features, thereby promoting mechanistic understanding of model behavior. This project builds on recent research from Anthropic, OpenAI, and Google DeepMind, providing a comprehensive pipeline for data capture, SAE training, and feature analysis.
Amazon to invest another $4B in Anthropic, OpenAI's biggest rival
Amazon's latest $4 billion investment in Anthropic raises its total stake to $8 billion, solidifying its role as a key player in the AI sector while remaining a minority investor.
Do Large Language Models learn world models or just surface statistics? (2023)
Large Language Models (LLMs) may develop world models rather than merely memorizing surface statistics, as evidenced by the Othello-GPT's ability to make legal moves with an error rate of 0.01% after training on game scripts, compared to 93.29% for the untrained model.
OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
OpenScholar is a retrieval-augmented language model that synthesizes scientific literature by extracting relevant passages from 45 million open-access papers, providing citation-backed responses to queries.
WhisperNER: Unified Open Named Entity and Speech Recognition
WhisperNER integrates named entity recognition (NER) with automatic speech recognition (ASR), enhancing both transcription accuracy and informativeness through joint processing of speech and entities.
A “meta-optics” camera that is the size of a grain of salt
The meta-optics camera, developed by researchers at Princeton University and the University of Washington, is 500,000 times smaller than traditional cameras while producing full-color images of comparable quality, revolutionizing imaging technology.
DOJ proposal would require Google to divest from AI partnerships with Anthropic
Google's $2 billion investment in Anthropic is jeopardized as the DOJ seeks to unwind the deal amid ongoing antitrust scrutiny over Google's dominance in online search.
Google’s recent breakthrough in Quantum Error Correction (QEC) demonstrates that logical qubits can outperform physical qubits, achieving a 2.14-fold reduction in logical error rates by increasing code distance from five to seven, showcasing the effectiveness of surface codes.
[R] Geometric aperiodic fractal organization in Semantic Space: A Novel Finding About How Meaning Organizes Itself
Significant discovery reveals that meaning in semantic space exhibits consistent geometric patterns across various dimensionality reduction techniques and embedding models, suggesting an inherent structure in how meaning organizes itself.
NeuralDEM – Real-Time Simulation of Industrial Particulate Flows
NeuralDEM revolutionizes the discrete element method (DEM) by replacing traditional, slow numerical simulations with fast, adaptable deep learning surrogates, enabling efficient modeling of complex fluid-mechanical systems.
Batched reward model inference and Best-of-N sampling
Batched reward model inference enhances efficiency in reinforcement learning, particularly for applications like tree search and MCTS, where traditional methods struggle with high throughput.
[R] BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
BALROG is a new benchmark aimed at enhancing agentic capabilities in LLMs and VLMs, promising to push the boundaries of AI reasoning in gaming contexts.
[R] Entropy-Guided Critical Neuron Pruning for Efficient Spiking Neural Networks
This paper presents a novel pruning method for Spiking Neural Networks (SNNs) that leverages neuronal avalanche analysis to pinpoint critical neurons, achieving 90% compression while preserving accuracy on datasets like MNIST and CIFAR-10.
Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
Hallucinations in large language models stem from their ability to recognize entities, revealing that models possess self-knowledge about their recall capabilities, which can influence their responses.
Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective
Large Language Models (LLMs) are transforming Natural Language Processing (NLP) but struggle with hallucinations, producing plausible yet incorrect outputs, which jeopardizes their trustworthiness and application across various fields.