# Dec 2, 2024

### World Labs: Generate 3D worlds from a single image
- **AI system generates 3D worlds** from a single image, marking a significant advancement in **spatial intelligence** technology.

### What happens if we remove 50 percent of Llama?
- **Sparse Llama 3.1 8B** is a **50% pruned** version of Meta's Llama 3.1, achieving **98% accuracy recovery** on the Open LLM Leaderboard and full recovery across various fine-tuning tasks, demonstrating its efficiency in maintaining performance while reducing model size.

### Procedural knowledge in pretraining drives reasoning in large language models
- **Procedural knowledge** in pretraining significantly enhances reasoning capabilities in Large Language Models (LLMs), revealing that models utilize distinct data sets for factual versus reasoning tasks, with procedural documents being crucial for the latter.

### KlongPy: High-Performance Array Programming in Python
- **KlongPy** is a high-performance Python adaptation of the **Klong array language**, designed for efficient vectorized operations using **NumPy** and supporting both **CPU and GPU** backends through **CuPy**.

### [R] A Comprehensive Database of 300+ Production LLM Implementations with Technical Architecture Details
- A **newly released database** catalogs over **300 real-world LLM implementations**, detailing their **technical architectures** and engineering decisions, providing a rich resource for ML practitioners.

### Unlocking the power of time-series data with multimodal models
- **Multimodal models significantly enhance time-series data understanding** by utilizing visual plots, achieving performance improvements of up to **120%** in classification tasks like fall detection compared to numerical data representation.

### [R] Simplified RNNs Achieve Transformer-Like Performance with Parallel Training and Reduced Parameters
- **RNNs can achieve transformer-like performance** on NLP tasks, particularly in language modeling, when employing the novel "RNN with Parallel Generation" (RPG) technique, which allows for token generation in parallel, enhancing efficiency.

### [P] Promptwright - Open source project to generate large synthetic datasets using an LLM (local or hosted)
- **Promptwright** is an **open source** tool that enables the generation of synthetic datasets using various **large language models (LLMs)**, either locally or through hosted services like OpenAI and Google Gemini, enhancing accessibility for developers.

### Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing
- **Video-Ma$^2$mba** introduces a novel architecture that integrates **State Space Models (SSMs)** within the Mamba-2 framework, enabling **linear scaling** of memory and computational demands for long video sequences.

### 🤗Investing in Performance: Fine-tune small models with LLM insights - a CFM case study
- **Fine-tuning small models** with insights from large language models (LLMs) can enhance Named Entity Recognition (NER) accuracy by up to **6.4%**, while reducing operational costs to **80x cheaper** than using large LLMs alone, as demonstrated in Capital Fund Management's case study.

### 🤗Open Source Developers Guide to the EU AI Act
- The **EU AI Act** introduces comprehensive regulations for AI, impacting open source developers by requiring clear documentation and compliance with existing copyright and privacy laws, particularly for **general purpose AI (GPAI)** models.

### A Simple and Provable Scaling Law for the Test-Time Compute of Large Language Models
- The proposed **two-stage algorithm** for large language models (LLMs) generates **N candidate solutions** and selects the best through a **multiple-round knockout tournament**, requiring **N × (K + 1)** parallelizable LLM calls for problem-solving.

### HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
- **HadaCore** is a **Hadamard Transform CUDA kernel** that achieves **1.1–1.4x** speedup on NVIDIA A100 and **1.0–1.3x** on H100 GPUs, with peak gains of **3.5x** and **3.6x** over existing Fast Hadamard implementations, enhancing model inference speeds while reducing quantization errors.