Dola Decoding by Contrasting Layers Improves Factuality in Large Language Models
DoLa, a novel decoding strategy, significantly reduces hallucinations in large language models by contrasting logits from different transformer layers, enhancing factuality without external knowledge or fine-tuning.
Turbopuffer: Fast search on object storage
turbopuffer is a cost-efficient, high-performance search engine that leverages object storage and smart caching, designed to scale to billions of vectors and millions of tenants/namespaces, addressing the high costs and operational challenges of existing search solutions.
Vision language models are blind
VLMs struggle with basic visual tasks such as identifying overlapping circles or counting objects, tasks trivial for humans.
MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
MobileLLM introduces optimizations for sub-billion parameter language models for on-device use, integrating SwiGLU activation, deep and thin architectures, embedding sharing, and grouped-query attention, achieving up to 4.3% accuracy improvement over state-of-the-art models on zero-shot commonsense reasoning tasks. Read the paper
RouteLLM: A framework for serving and evaluating LLM routers
RouteLLM introduces a framework for efficiently routing queries between large language models (LLMs), achieving up to an 85% reduction in costs while maintaining 95% of GPT-4's performance, as detailed in their paper.
Knowledge Graphs in RAG: Hype vs. Ragas Analysis
Knowledge graphs in RAG, when analyzed, showed similar context relevancy scores (~0.74) across different methods, including FAISS, suggesting that the presence of a knowledge graph does not inherently improve context retrieval.
Training of Physical Neural Networks
Physical Neural Networks (PNNs) leverage the properties of physical systems to perform computation, potentially enabling AI models 1000x larger than current ones to operate locally and privately on edge devices.
From GPT-4 to AGI: Counting the OOMs
AGI by 2027 is strikingly plausible, with advancements from GPT-2 to GPT-4 suggesting a similar leap in capabilities could occur within the next four years, driven by trends in compute, algorithmic efficiencies, and "unhobbling" gains.
[R] Scaling Synthetic Data Creation with Personas
The Persona Hub introduces a novel persona-driven data synthesis methodology, leveraging 1 billion diverse personas to create synthetic data across various scenarios, enhancing the versatility and scalability of data generation.
[D] Inference-time gradients
Diffusion models excel in inference-time gradients by predicting score/noise/gradient to reverse the diffusion process, notably through rectified flow in Stable Diffusion 3.
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
The Lookback Lens method detects contextual hallucinations in LLMs by analyzing the ratio of attention weights on provided context versus newly generated tokens, revealing a novel approach to understanding and mitigating inaccuracies in model responses.
Metron: Holistic Performance Evaluation Framework for LLM Inference Systems
Metron introduces a novel metric, fluidity-index, to better evaluate LLM inference systems, addressing the inadequacies of traditional performance metrics like latency and throughput.
[R] How GraphRAG works? Explained
GraphRAG enhances RAG (Retrieval-Augmented Generation) by improving the connection of external documents to Large Language Models (LLMs), facilitating more informed and context-rich responses.
FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
The FBI-LLM introduces a novel approach to training fully binarized large language models from scratch, achieving parity with full-precision models without the need for pretrained weights.
Virtual Personas for Language Models via an Anthology of Backstories
Anthology enhances LLMs by conditioning them with virtual personas through open-ended life narratives, improving their utility in behavioral studies.