GameNGen, the first game engine powered entirely by a neural model, enables real-time interaction with complex environments, simulating the classic game DOOM at over 20 frames per second on a single TPU.
Writing in the Margins (WiM)
Writing in the Margins (WiM) introduces a new inference pattern for Large Language Models, optimizing long input sequence handling through chunked prefill and segment-wise inference.
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
Eagle explores the design space for multimodal large language models (MLLMs) by focusing on mixture of vision encoders, revealing that simple concatenation of visual tokens from various encoders is as effective as complex strategies.
Enhanced Visual Perception in MLLMs
Enhanced visual perception in multimodal large language models ( MLLMs) significantly reduces hallucinations and improves performance in tasks like optical character recognition and document analysis, highlighting the importance of vision encoders.
Repo2vec – An Open-Source Library for Chatting with Codebases
repo2vec is a modular library designed for chatting with codebases, simplifying the process of understanding and integrating code without manual code review.
Rerun 0.18: Faster Ingestion and Lower Memory Use
Rerun 0.18 significantly improves ingestion speed and memory efficiency for large time-series datasets by exploiting column chunks, achieving 100x faster write and ingestion speed and 35x lower memory overhead.
NVIDIA Blackwell Sets New Standard for Generative AI
NVIDIA Blackwell GPU significantly outperforms its predecessor, the NVIDIA H100 Tensor Core GPU, by delivering up to 4x more performance on the Llama 2 70B benchmark, leveraging advancements like a second-generation Transformer Engine and FP4 Tensor Cores.
AIExh: A Directory for Open Source AI Projects
AIExh serves as a global hub for open-source AI projects, featuring over 2720 new projects to foster innovation and collaboration within the AI community.
Meta's Sapiens for Human Vision Tasks
Meta's Sapiens significantly advances human vision tasks by integrating pose, segmentation, depth, and normal predictions into a unified model, as detailed in their paper.
Rotary Positional Embeddings (RoPE)
The Rotary Positional Embeddings (RoPE) mechanism enhances transformers by integrating positional information directly into the attention mechanism, offering a nuanced approach to understanding sequence order.
Pytorch Library for Signed Distance Function
The Pytorch library introduces volumetric data structures that seamlessly integrate with Pytorch, supporting gradients and automatic differentiation.
CBF-LLM: Safe Control for LLM Alignment
The CBF-LLM framework introduces a control barrier function to guide large language models towards generating text that aligns with user preferences, enhancing safety and relevance.
More Than Fine: Multi-LoRA Support in NVIDIA RTX AI Toolkit
The latest update to the NVIDIA RTX AI Toolkit introduces multi-LoRA support, enhancing the performance of fine-tuned large language models (LLMs) on RTX AI PCs and workstations by up to 6x.
TwoMinutePapers - Crazy New AI Learned To Rewrite Doom!