# Apr 6, 2024

## Language models are Super Mario: Absorbing abilities from homologous models

- **Language Models (LMs) can now absorb new abilities from similar models** through a novel technique called DARE, which simplifies the merging of capabilities without the need for retraining or advanced hardware.

## Long-form factuality in large language models

- **Large language models (LLMs)** often produce content with factual inaccuracies in response to open-ended, fact-seeking prompts, prompting the development of **LongFact**, a benchmark using GPT-4 across 38 topics to assess long-form factuality.

## AI and the Problem of Knowledge Collapse

- **AI's potential for vast data processing and productivity gains** is counterbalanced by the risk of **knowledge collapse**, a phenomenon where reliance on AI narrows public understanding and innovation.

## Fortran on WebAssembly

- **Dr. George W Stagg** demonstrates compiling **Fortran code to WebAssembly** using a patched LLVM's `flang-new` compiler, enabling Fortran routines to run in web browsers.

## Autonomous Overhead Powerline Recharging for Uninterrupted Drone Operations

- The research introduces a **drone capable of autonomously recharging** from overhead power lines, enabling **sustained, uninterrupted operations** over multiple hours by leveraging an onboard perception and autonomy component for power line detection and navigation.

## ML researchers who are not in NLP, what are you researching? Please share.

- **ML researchers** outside the NLP domain are invited to share their **research areas**, aiming for a comprehensive view of the **spectrum of ML research**.

## Loki: An open-source tool for fact verification

- **Loki is an open-source tool designed to automate fact verification**, providing a pipeline that includes decomposing texts into claims, assessing their significance, generating queries, crawling for evidence, and verifying claims, aimed at journalists and researchers.

## Chisel: A fast TCP/UDP tunnel over HTTP

- **Chisel** is a **fast TCP/UDP tunnel** over HTTP, secured with SSH, combining both client and server functionalities into a single Go executable, designed for firewall traversal and secure network entry.

## How do you get enough patience to train-debug-train models?

- **Patience in model training** is tested when errors emerge after long periods of waiting, highlighting the **frustration** of paused tasks and wasted time.

## Tokens, N-Grams, and Bag-of-Words Models

- **Tokens and n-grams** serve as foundational elements in Natural Language Processing (NLP), enabling the understanding and manipulation of text by breaking it down into manageable units for analysis.

## PID Controller Explainer (2022)

- **PID controllers** adjust system inputs based on the difference between a desired set point and an actual sensor measurement, optimizing for minimal error without requiring knowledge of the system's internal mechanics.

## Schedule-Free Learning – A New Way to Train

- **Schedule-Free learning** employs a novel approach by replacing the momentum of an underlying optimizer with interpolation and averaging, facilitating **faster training without the need for predefined schedules**.

## Optuna – A Hyperparameter Optimization Framework

- **Optuna** is an open-source framework designed for the automation of hyperparameter optimization, featuring **eager search spaces, state-of-the-art algorithms, and easy parallelization**.

## I just can't fine tune BERT over 40% accuracy for text-classification task

- The **initial struggle** to fine-tune BERT for a text-classification task with **40% accuracy** was overcome by addressing **data insufficiency and class imbalance**, leading to an **88.45% accuracy** on the validation set.

## Photoshop for Text (2022)

- **Text editing** is evolving beyond basic functions like cut and paste to include complex transformations, such as altering style or summarizing content, akin to **image manipulation** in Photoshop.