# May 27, 2024

## Daily

- **DiscoGrad** enhances **Automatic Differentiation (AD)** for **branchy programs** by calculating smoothed gradients across branches, addressing the challenge of unhelpful gradients in parameter-dependent branching and randomness.

- **Transcription Stream Community Edition** offers a **self-hosted, offline diarization service** with features like drag-and-drop transcription, web interface for file management, and full-text search capabilities, enhanced by **Ollama and Mistral** for summarization.

- **xAI secures a Series B funding round of $6 billion**, with contributions from notable investors like Valor Equity Partners and Andreessen Horowitz, aiming to advance its AI technologies.

- **Artificial Intelligence (AI) is posited to decrease the skill premium** by being more substitutable for tasks performed by high-skill workers compared to the substitutability of low-skill workers for high-skill tasks.

- **TabForestPFN**, a novel **in-context learning transformer**, outperforms traditional tree-based algorithms in **tabular data classification** by incorporating a fine-tuning stage and a synthetic data generator.

- **AstroPT**, an **autoregressive pretrained transformer**, is tailored for **astronomy**, leveraging **8.6 million $512 \times 512$ pixel $grz$-band galaxy postage stamp observations** from the **DESI Legacy Survey DR8** to train models ranging from **1 million to 2.1 billion parameters**.

- **MOMENT** is a **new foundation model** designed to tackle a variety of **time-series tasks**, including forecasting, classification, anomaly detection, and imputation, marking a significant advancement in time-series analysis.

- **Model growth**, specifically through a **depthwise stacking operator** ($G_{\text{stack}}$), significantly **accelerates LLM pre-training**, showcasing **reduced loss** and **enhanced performance** across **eight NLP benchmarks**.

- **Closed-source foundation models** are poised to dominate due to **scaling laws and centralizing forces**, leaving open-source alternatives behind as they become financially unsustainable and less competitive.

- **NuwaTS** introduces a **novel framework** that repurposes Pre-trained Language Models (PLMs) for **time series imputation** across **any domain** and **missing patterns**, leveraging **specific embeddings** and a **contrastive learning approach**.

- The **relationship between environment complexity and optimal policy convergence** in reinforcement learning (RL) is under exploration, focusing on how intricate environments impact the learning and effectiveness of policies.

- **NVIDIA and the University of Utah** have developed a **new ray tracing technique** that significantly reduces noise, making real-time, beautifully ray-traced images a reality.

- **Machine unlearning** aims to **selectively forget or reduce undesirable knowledge** in large language models (LLMs), focusing on **ethical, privacy, and safety standards** through gradient ascent algorithm application.

- **$i$REPO** introduces a **novel LLM alignment framework** that leverages **implicit Reward pairwise difference regression** for Empirical Preference Optimization, addressing the challenge of aligning large language models with human expectations by mitigating untruthful, toxic, or biased outputs.

- The **Dynamical Systems Framework (DSF)** introduces a unified approach to compare and understand **softmax attention**, **linear attention**, **State Space Models (SSMs)**, and **Recurrent Neural Networks (RNNs)**, highlighting their efficiency and scalability in AI applications.