LLMs can play chess surprisingly well despite not being designed for it, demonstrating an unexpected ability to interpret game states from textual move lists, hinting at their potential for broader applications in complex tasks.
Omnivision-968M: Vision Language Model with 9x Tokens Reduction for Edge Devices
OmniVision-968M is a compact multimodal model with 968 million parameters, designed for efficient processing of visual and text inputs on edge devices, achieving a 9x reduction in image tokens from 729 to 81, which significantly lowers latency and computational costs.
BERTs Are Generative In-Context Learners
BERTs, specifically DeBERTa, can perform generative tasks through a simple inference technique, showcasing that masked language models possess in-context learning capabilities akin to causal models like GPT.
[D] Paper Club: Nvidia Researcher Ethan He Presents Upcycling LLMs in MoE
Ethan He from Nvidia will present his research on Upcycling LLMs in Mixture of Experts (MoE), showcasing innovative techniques for enhancing large language models.
[R] Undetectable Backdoors in ML Models: Novel Techniques Using Digital Signatures and Random Features, with Implications for Adversarial Robustness
Undetectable backdoors can be inserted into ML models using digital signatures and random features, enabling malicious manipulation of outputs without detection, even under white-box conditions.
[Discussion] Scaling laws and graph neural networks
The introduction of the first "graph foundation model" demonstrates that graph neural networks (GNNs) can effectively scale with both data and model size, offering robust generalization across various domains and efficient fine-tuning capabilities on new datasets, as detailed in the arXiv paper.
[R] The geometry of data: the missing metric tensor and the Stein score
The article discusses how to derive an efficient metric tensor for data manifolds using the Stein score, highlighting its significance in the context of differential geometry and machine learning.
Keeping an AI on Diabetes Risk: Gen AI Model Predicts Blood Sugar Levels Four Years Out
GluFormer, a generative AI model developed by researchers from the Weizmann Institute, Pheno.AI, and NVIDIA, can predict an individual's glucose levels and health metrics up to four years in advance using past monitoring data.
GraphRAG: Improving global search via dynamic community selection
GraphRAG enhances global search by implementing dynamic community selection, which prunes irrelevant reports early, leading to a 77% reduction in token costs while maintaining response quality comparable to static methods.
Expanding Our Collaboration with Amazon: Stable Diffusion 3.5 Large is Now Available in Amazon SageMaker JumpStart
Stable Diffusion 3.5 Large, now available in Amazon SageMaker JumpStart, is designed for startups and enterprises to generate high-quality images efficiently, leveraging 8 billion parameters for superior prompt accuracy and image quality.
LLM Hallucination Reasoning with Zero-shot Knowledge Test
LLM hallucination presents a challenge in generating accurate text, necessitating a new classification system that identifies hallucinations as aligned, misaligned, or fabricated to enhance detection methods.
On the Limits of Language Generation: Trade-Offs Between Hallucination and Mode Collapse
Language models face a fundamental trade-off between hallucination (outputting invalid strings) and mode collapse (failing to capture the full range of a language), making it challenging to meet both requirements simultaneously.
StreamAdapter: Efficient Test Time Adaptation from Contextual Streams
StreamAdapter introduces a method for efficient test time adaptation of large language models (LLMs) by directly updating model parameters from contextual streams, thus eliminating the need for explicit in-context demonstrations.