# Aug 29, 2024

## Diffusion Models are Real-Time Game Engines

- **GameNGen**, the **first game engine powered entirely by a neural model**, enables real-time interaction with complex environments, simulating the classic game **DOOM** at over **20 frames per second** on a single TPU.

## Writing in the Margins (WiM)

- **Writing in the Margins (WiM)** introduces a **new inference pattern** for Large Language Models, optimizing long input sequence handling through **chunked prefill** and segment-wise inference.

## Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

- **Eagle** explores the **design space for multimodal large language models (MLLMs)** by focusing on **mixture of vision encoders**, revealing that **simple concatenation of visual tokens** from various encoders is as effective as complex strategies.

## Enhanced Visual Perception in MLLMs

- **Enhanced visual perception** in multimodal large language models ( **MLLMs**) significantly **reduces hallucinations** and **improves performance** in tasks like optical character recognition and document analysis, highlighting the importance of **vision encoders**.

## Repo2vec – An Open-Source Library for Chatting with Codebases

- **`repo2vec` is a modular library designed for chatting with codebases**, simplifying the process of understanding and integrating code without manual code review.

## Rerun 0.18: Faster Ingestion and Lower Memory Use

- **Rerun 0.18** significantly **improves ingestion speed and memory efficiency** for large time-series datasets by **exploiting column chunks**, achieving **100x faster write and ingestion speed** and **35x lower memory overhead**.

## NVIDIA Blackwell Sets New Standard for Generative AI

- **NVIDIA Blackwell GPU** significantly outperforms its predecessor, the NVIDIA H100 Tensor Core GPU, by delivering **up to 4x more performance** on the Llama 2 70B benchmark, leveraging advancements like a second-generation Transformer Engine and FP4 Tensor Cores.

## AIExh: A Directory for Open Source AI Projects

- **AIExh** serves as a **global hub for open-source AI projects**, featuring over **2720 new projects** to foster innovation and collaboration within the AI community.

## Meta's Sapiens for Human Vision Tasks

- **Meta's Sapiens** significantly advances **human vision tasks** by integrating pose, segmentation, depth, and normal predictions into a unified model, as detailed in their [paper](https://arxiv.org/abs/2408.12569).

## Rotary Positional Embeddings (RoPE)

- The **Rotary Positional Embeddings (RoPE)** mechanism enhances **transformers** by integrating positional information directly into the attention mechanism, offering a nuanced approach to understanding sequence order.

## Pytorch Library for Signed Distance Function

- The **[Pytorch library](https://github.com/UM-ARM-Lab/pytorch_volumetric)** introduces **volumetric data structures** that seamlessly integrate with Pytorch, supporting gradients and automatic differentiation.

## CBF-LLM: Safe Control for LLM Alignment

- The **CBF-LLM framework** introduces a **control barrier function** to guide **large language models** towards generating text that aligns with **user preferences**, enhancing safety and relevance.

## More Than Fine: Multi-LoRA Support in NVIDIA RTX AI Toolkit

- The **latest update** to the NVIDIA RTX AI Toolkit introduces **multi-LoRA support**, enhancing the performance of fine-tuned large language models (LLMs) on RTX AI PCs and workstations by **up to 6x**.

## TwoMinutePapers - Crazy New AI Learned To Rewrite Doom!