# Jul 10, 2024

## Dola Decoding by Contrasting Layers Improves Factuality in Large Language Models
- **DoLa**, a novel decoding strategy, **significantly reduces hallucinations** in large language models by contrasting logits from different transformer layers, **enhancing factuality** without external knowledge or fine-tuning.

## Turbopuffer: Fast search on object storage
- **turbopuffer** is a **cost-efficient, high-performance search engine** that leverages **object storage and smart caching**, designed to scale to billions of vectors and millions of tenants/namespaces, addressing the high costs and operational challenges of existing search solutions.

## Vision language models are blind
- **VLMs struggle with basic visual tasks** such as identifying overlapping circles or counting objects, tasks trivial for humans.

## MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
- **MobileLLM** introduces **optimizations for sub-billion parameter language models** for on-device use, integrating SwiGLU activation, deep and thin architectures, embedding sharing, and grouped-query attention, achieving **up to 4.3% accuracy improvement** over state-of-the-art models on zero-shot commonsense reasoning tasks. [Read the paper](https://arxiv.org/abs/2402.14905)

## RouteLLM: A framework for serving and evaluating LLM routers
- **RouteLLM introduces a framework for efficiently routing queries between large language models (LLMs), achieving up to an 85% reduction in costs while maintaining 95% of GPT-4's performance**, as detailed in their [paper](https://arxiv.org/abs/2406.18665).

## Knowledge Graphs in RAG: Hype vs. Ragas Analysis
- **Knowledge graphs in RAG, when analyzed, showed similar context relevancy scores (~0.74) across different methods, including FAISS, suggesting that the presence of a knowledge graph does not inherently improve context retrieval.**

## Training of Physical Neural Networks
- **Physical Neural Networks (PNNs)** leverage the properties of physical systems to **perform computation**, potentially enabling AI models **1000x larger** than current ones to operate **locally and privately on edge devices**.

## From GPT-4 to AGI: Counting the OOMs
- **AGI by 2027 is strikingly plausible**, with advancements from GPT-2 to GPT-4 suggesting a similar leap in capabilities could occur within the next four years, driven by trends in compute, algorithmic efficiencies, and "unhobbling" gains.

## [R] Scaling Synthetic Data Creation with Personas
- The **Persona Hub** introduces a **novel persona-driven data synthesis methodology**, leveraging **1 billion diverse personas** to create synthetic data across various scenarios, enhancing the versatility and scalability of data generation.

## [D] Inference-time gradients
- **Diffusion models** excel in **inference-time gradients** by predicting **score/noise/gradient** to reverse the diffusion process, notably through **[rectified flow](https://arxiv.org/abs/2209.03003)** in **[Stable Diffusion 3](https://arxiv.org/abs/2403.03206)**.

## Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
- The **Lookback Lens** method detects **contextual hallucinations** in LLMs by analyzing the **ratio of attention weights** on provided context versus newly generated tokens, revealing a novel approach to understanding and mitigating inaccuracies in model responses.

## Metron: Holistic Performance Evaluation Framework for LLM Inference Systems
- **Metron** introduces a **novel metric, fluidity-index**, to better evaluate **LLM inference systems**, addressing the inadequacies of traditional performance metrics like latency and throughput.

## [R] How GraphRAG works? Explained
- **GraphRAG** enhances **RAG (Retrieval-Augmented Generation)** by improving the connection of external documents to **Large Language Models (LLMs)**, facilitating more informed and context-rich responses.

## FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
- The **FBI-LLM** introduces a **novel approach** to training **fully binarized large language models** from scratch, achieving parity with full-precision models without the need for pretrained weights.

## Virtual Personas for Language Models via an Anthology of Backstories
- **Anthology** enhances **LLMs** by conditioning them with **virtual personas** through **open-ended life narratives**, improving their utility in behavioral studies.