Jul 11, 2024

Daily

• DoLa Decoding by Contrasting Layers Improves Factuality in Large Language Models
  DoLa, a novel decoding strategy, significantly reduces hallucinations in large language models by contrasting logits from different transformer layers, enhancing factuality without external knowledge or fine-tuning.

• Vision language models are blind
  VLMs struggle with basic visual tasks such as identifying overlapping circles or counting objects, tasks trivial for humans.

• Turbopuffer: Fast search on object storage
  turbopuffer is a cost-efficient, high-performance search engine that leverages object storage and smart caching, designed to scale to billions of vectors and millions of tenants/namespaces, addressing the high costs and operational challenges of existing search solutions.

• Inference Performance Optimization for Large Language Models on CPUs
  Optimizing inference performance for Large Language Models (LLMs) on CPUs addresses the challenge of deploying high-performance LLMs in low-resource environments, focusing on reducing financial and hardware constraints.

• RouteLLM: A framework for serving and evaluating LLM routers
  RouteLLM introduces a framework for efficiently routing queries between large language models (LLMs), achieving up to an 85% reduction in costs while maintaining 95% of GPT-4's performance, as detailed in their paper.

• FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-Precision
  FlashAttention-3 introduces techniques exploiting asynchrony and low-precision (FP8) on Hopper GPUs, achieving 1.5-2.0x speed improvements and 75% utilization of H100 GPU capabilities, significantly enhancing the efficiency of attention mechanisms in large language models (LLMs).

• Karpathy: Let's reproduce GPT-2 (1.6B): one 8XH100 node 24h $672 in llm.c
  Reproducing GPT-2 (1.6B parameters) using llm.c on a single 8XH100 node took 24 hours and cost $672, showcasing significant advancements in compute, software, and data availability since its initial release by OpenAI in 2019.

• Training of Physical Neural Networks
  Physical Neural Networks (PNNs) leverage the properties of physical systems to perform computation, potentially enabling AI models 1000x larger than current ones to operate locally and privately on edge devices.

• Korvus: Single-Query RAG with Postgres
  Korvus integrates the entire RAG pipeline into a single Postgres database query, offering a unified search SDK with Python, JavaScript, and Rust bindings for high-performance, customizable search functionalities.

• GitHub Copilot is not infringing your copyright
  GitHub Copilot, trained on publicly available source code, does not infringe copyright by generating code suggestions, despite using GPL-licensed repositories for training.

• From Unlabeled Data to Rich Segmentation: The Magic of Self-Supervised Models
  Self-supervised learning with DINOv2 ViT weights from Facebook Research enables rich image segmentation by finetuning with Low-Rank Adaptation (LoRA) and simple decoders, achieving solid validation IoU scores.

• Memory^3: Language Modeling with Explicit Memory
  Memory^3 introduces a novel approach to language modeling by equipping large language models (LLMs) with explicit memory, significantly reducing parameter size, training, and inference costs while outperforming larger models and text retrieval-augmented generation (RAG) models. Read the paper

• Controllable Navigation Instruction Generation with Chain of Thought Prompting
  C-Instructor leverages chain-of-thought-style prompting to enable style-controllable and content-controllable navigation instruction generation, addressing limitations of existing models by incorporating landmark identification for clearer guidance.

• TwoMinutePapers - NVIDIA’s New Tech Runs A Virtual City!
  NVIDIA's breakthrough allows for the creation of virtual worlds from 250,000 photos, synthesizing unseen parts between images using Neural Radiance Fields (NERFs), enabling unprecedentedly large and detailed virtual scenes.