# May 12, 2024

- **Page Dewarping (2016)**  
  **Page dewarping** is achieved by transforming curled page images into flat versions, using a **parametric model** that simulates the page's 3D shape and orientation.

- **Vision Transformers Need Registers**  
  **Artifacts in Vision Transformer (ViT) feature maps**, identified in both supervised and self-supervised networks, manifest as high-norm tokens in low-informative background areas, impacting image processing quality.

- **Automatically Detecting Under-Trained Tokens in Large Language Models**  
  **Researchers have developed methods to automatically detect under-trained tokens**, like the notorious SolidGoldMagikarp, in Large Language Models (LLMs), addressing a significant gap in tokenizer and model training alignment.

- **Impact of solar storm on QLORA + RLHF of Llama3 8B?**  
  The **solar storm** has prompted NOAA to issue warnings about potential **damage to transformers** and **grid systems**, which could indirectly affect the operation of models like **Llama3 8B**.

- **Scientists Find an 'Alphabet' in Whale Songs**  
  Scientists have discovered a **"sperm whale phonetic alphabet"** in the click patterns of sperm whales, suggesting a complex communication system akin to human language.

- **Citation Needed – Wikimedia Foundation's Experimental LLM/RAG Chrome Extension**  
  The **Wikipedia Citation Needed** Chrome extension, developed by the Wikimedia Foundation, leverages **ChatGPT** to verify the reliability of online information by finding relevant Wikipedia citations.

- **Show NH: "data-to-paper" - autonomous stepwise LLM-driven research**  
  **The _data-to-paper_ framework** leverages AI, specifically LLM and rule-based agents, to automate the scientific research process from **data analysis to generating human-verifiable research papers**, enhancing transparency and verifiability. [arXiv:2404.17605](https://arxiv.org/abs/2404.17605)

- **Homoiconic Python**  
  Mohammed Jamal explores the concept of **homoiconicity** in Python by emulating Lisp's unique characteristic where **code and data are interchangeable**, providing a fresh perspective on programming language design.

- **Yi 1.5**  
  **Yi-1.5** is an **enhanced AI model** over its predecessor, boasting improvements in **coding, math, reasoning, and instruction-following**, while retaining its prowess in **language understanding and commonsense reasoning**.

- **Marcus Hutter's work on Universal Artificial Intelligence**  
  **Marcus Hutter** has authored **two seminal books** on Universal Artificial Intelligence (UAI), laying a foundation for **combining sequential prediction with action** to address challenges faced by intelligent agents in unknown environments.

- **AI Ruined Quora**  
  **Quora's decline** is attributed to an **AI-driven prompt generator** that produces bizarre and nonsensical questions, moving away from its original expert-driven content.

- **LoRA from scratch implementation for LLM classifier training**  
  **LoRA (Low-rank Adaptation) significantly reduces computational cost and time** for finetuning large models on specific tasks by adjusting a small subset of the model's parameters.

- **ENIAC programmer Jean Bartik's Amazon book review**  
  The review acknowledges the effort behind the work but does not consider it the ultimate resource on the subject.

- **Show HN: Multilspy – Cross platform framework to develop Language Server Clients**  
  **Microsoft's [monitors4codegen](https://github.com/microsoft/monitors4codegen) introduces Monitor-Guided Decoding (MGD)**, leveraging static analysis to enhance code generation by Language Models (LMs), guiding them to produce compilable and contextually accurate code.

- **Open source library to scrape PDFs, YouTube, URLs, Presentations, etc for API-hosted vision-language models**  
  The **open source library** [thepi.pe](https://thepi.pe/) enables **extraction of text and visual data** from diverse sources like PDFs, webpages, and YouTube videos, specifically designed for seamless integration with **API-hosted vision language models** such as GPT-4-Vision or Gemini.