# Nov 19, 2024

## ML Times

### Highlights

- **Llama 3.1 405B** achieves a groundbreaking **969 tokens/s** on Cerebras Inference, marking it as the **fastest frontier model** globally, outperforming competitors like GPT-4o by **12x** and Claude 3.5 Sonnet by **18x**.

- **AMD has surpassed Nvidia** in compute power on the Top500 list, primarily due to the performance of the "El Capitan" supercomputer, which achieved a peak theoretical performance of **2,746.4 petaflops** with its AMD MI300A devices.

- **LLaVA-o1** introduces a **novel approach** to Vision-Language Models (VLMs) by enabling **autonomous multistage reasoning**, significantly enhancing performance in complex visual question-answering tasks.

- **Fei-Fei Li's creation of ImageNet**, a dataset with 14 million images across 22,000 categories, was pivotal in demonstrating the potential of neural networks, despite initial skepticism from peers. This massive dataset provided the necessary training data that enabled deep learning models to excel in image recognition tasks.

- **Machine learning breakthroughs** from the past year have significantly advanced areas like natural language processing and computer vision, showcasing innovative algorithms and applications that enhance performance and efficiency.

- The **positional popcount** (pospopcnt) technique enhances byte histogramming by efficiently counting bits per position, leveraging **AVX512** and **GF2P8AFFINEQB** for optimized performance.

- **Batched reward model inference** enhances efficiency in reinforcement learning, particularly for applications like **tree search** and **MCTS**, where traditional methods struggle with high throughput.

- **NVLink** offers the **highest bandwidth** (up to **900 GB/s**) for GPU communication, but incurs **higher latency**, making it essential to evaluate workload needs when selecting interconnects.

- A **comprehensive collection** of open-source voice-cloning TTS models has been released, potentially the largest of its kind, available at [GitHub](https://github.com/ttsds/datasets).

- **Verifier engineering** introduces a **novel post-training paradigm** that enhances foundation models by utilizing automated verifiers for effective feedback and verification tasks.

- **Judge Arena** is a new platform that allows users to **benchmark LLMs** as evaluators through crowdsourced voting, enhancing the understanding of which models excel in judging capabilities.

- **NVIDIA's innovations in AI and supercomputing** are set to revolutionize industries, with tools for drug discovery, climate forecasting, and quantum simulations, showcasing a commitment to driving scientific breakthroughs.

- **NVIDIA's new NIM microservices** enable a **500x speedup** in delivering high-resolution weather simulations, enhancing AI model deployment for predicting extreme weather events like snow, ice, and hail.

- **cuPyNumeric** allows researchers to run their existing Python code on **thousands of GPUs** without any modifications, significantly enhancing data processing speed and efficiency across various scientific fields.

- The **NVIDIA H200 NVL PCIe GPU** delivers **1.7x faster** large language model inference and **1.3x improved** performance for high-performance computing applications, making it a powerful choice for enterprise data centers.