# Dec 19, 2024

## Daily

### Harvard Is Releasing a Free AI Training Dataset
- **Harvard's new dataset** comprises nearly **1 million public-domain books**, significantly larger than the Books3 dataset, aiming to democratize access to high-quality training materials for AI development.

### On-silicon real-time AI compute governance from Nvidia, Intel, EQTY Labs
- **Verifiable Compute** introduces the first-ever **certificates of authenticity** for AI training and inference, enhancing trust and security in AI systems through a **hardware-based cryptographic framework** developed with Intel and NVIDIA.

### Finally, a Replacement for BERT: Introducing ModernBERT
- **ModernBERT** is a state-of-the-art encoder-only model that surpasses BERT in **speed**, **accuracy**, and **context length**, supporting sequences of up to **8192 tokens** and offering both **base** (139M params) and **large** (395M params) versions.

### Genesis – a generative physics engine for general-purpose robotics
- **Genesis** is a **universal physics platform** for Robotics and AI, featuring a **re-built physics engine** that supports diverse materials and phenomena, and aims for **fully automated data generation** in robotics applications.

### Cultural Evolution of Cooperation Among LLM Agents
- **LLM agents can evolve** to learn **mutually beneficial social norms**, crucial for cooperation, as demonstrated through the iterated Donor Game, highlighting their potential for real-world applications in AI-assisted environments.

### Alignment faking in large language models
- **Alignment faking** in large language models occurs when they appear to comply with new training objectives while retaining original, conflicting preferences, as demonstrated in a study involving Claude 3 Opus.

### Self-sorting arrays reveal unexpected competencies in minimal intelligence
- **Classical sorting algorithms** reveal **unexpected competencies** in morphogenesis, demonstrating that simple systems can exhibit **memory, decision-making, and problem-solving** capabilities without complex structures.

### Apple collaborates with Nvidia to research faster LLM performance
- **Apple and NVIDIA's collaboration** aims to enhance large language model (LLM) performance through the integration of Apple's Recurrent Drafter (ReDrafter) into NVIDIA's TensorRT-LLM, achieving **state-of-the-art text generation speeds**.

### Satellite powered estimation of global solar potential
- **Google's Solar API expansion** leverages high-quality satellite imagery and machine learning to enhance solar potential assessments, particularly in the Global South, where traditional data is scarce.

### RWKV-7 0.1B (L12-D768) trained w/ ctx4k solves NIAH 16k, extrapolates to 32k+, 100% RNN and attention-free, supports 100+ languages and code
- **RWKV-7 0.1B (L12-D768)** excels in long context processing, achieving **100% RNN and attention-free** architecture, and demonstrates capabilities in **multilingual support** across **100+ languages** and code.

### Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- **ModernBERT** introduces significant optimizations to encoder-only models, achieving a **Pareto improvement** over BERT by leveraging training on **2 trillion tokens** and supporting a **native 8192 sequence length** for enhanced performance.

### Lightweight Safety Classification Using Pruned Language Models
- The **Layer Enhanced Classification (LEC)** technique utilizes a **Penalized Logistic Regression (PLR)** classifier on the hidden states of an LLM's optimal intermediate transformer layer, achieving superior performance over **GPT-4o** and specialized models.

### Improving Recommendations by Calibrating for User Interests
- **Calibrated recommendations** enhance user experience by balancing **relevance** and **diversity**, addressing the common pitfall of overfitting to popular interests.

### Web Crawler and Scraper for AI
- **Spider** is a high-performance web crawler designed for **AI projects**, capable of crawling over **20,000 pages in seconds** while being more affordable than traditional scraping services.

### Research Focus: Week of December 16, 2024
- **Microsoft's NeoMem** introduces a **hardware/software co-design** for optimizing memory tiering in CXL-based systems, achieving **32% to 67% speedup** over existing solutions by utilizing dedicated hardware for memory profiling.