May 12, 2024

• Page Dewarping (2016)
  Page dewarping is achieved by transforming curled page images into flat versions, using a parametric model that simulates the page's 3D shape and orientation.

• Vision Transformers Need Registers
  Artifacts in Vision Transformer (ViT) feature maps, identified in both supervised and self-supervised networks, manifest as high-norm tokens in low-informative background areas, impacting image processing quality.

• Automatically Detecting Under-Trained Tokens in Large Language Models
  Researchers have developed methods to automatically detect under-trained tokens, like the notorious SolidGoldMagikarp, in Large Language Models (LLMs), addressing a significant gap in tokenizer and model training alignment.

• Impact of solar storm on QLORA + RLHF of Llama3 8B?
  The solar storm has prompted NOAA to issue warnings about potential damage to transformers and grid systems, which could indirectly affect the operation of models like Llama3 8B.

• Scientists Find an 'Alphabet' in Whale Songs
  Scientists have discovered a "sperm whale phonetic alphabet" in the click patterns of sperm whales, suggesting a complex communication system akin to human language.

• Citation Needed – Wikimedia Foundation's Experimental LLM/RAG Chrome Extension
  The Wikipedia Citation Needed Chrome extension, developed by the Wikimedia Foundation, leverages ChatGPT to verify the reliability of online information by finding relevant Wikipedia citations.

• Show NH: "data-to-paper" - autonomous stepwise LLM-driven research
  The data-to-paper framework leverages AI, specifically LLM and rule-based agents, to automate the scientific research process from data analysis to generating human-verifiable research papers, enhancing transparency and verifiability. arXiv:2404.17605

• Homoiconic Python
  Mohammed Jamal explores the concept of homoiconicity in Python by emulating Lisp's unique characteristic where code and data are interchangeable, providing a fresh perspective on programming language design.

• Yi 1.5
  Yi-1.5 is an enhanced AI model over its predecessor, boasting improvements in coding, math, reasoning, and instruction-following, while retaining its prowess in language understanding and commonsense reasoning.

• Marcus Hutter's work on Universal Artificial Intelligence
  Marcus Hutter has authored two seminal books on Universal Artificial Intelligence (UAI), laying a foundation for combining sequential prediction with action to address challenges faced by intelligent agents in unknown environments.

• AI Ruined Quora
  Quora's decline is attributed to an AI-driven prompt generator that produces bizarre and nonsensical questions, moving away from its original expert-driven content.

• LoRA from scratch implementation for LLM classifier training
  LoRA (Low-rank Adaptation) significantly reduces computational cost and time for finetuning large models on specific tasks by adjusting a small subset of the model's parameters.

• ENIAC programmer Jean Bartik's Amazon book review
  The review acknowledges the effort behind the work but does not consider it the ultimate resource on the subject.

• Show HN: Multilspy – Cross platform framework to develop Language Server Clients
  Microsoft's monitors4codegen introduces Monitor-Guided Decoding (MGD), leveraging static analysis to enhance code generation by Language Models (LMs), guiding them to produce compilable and contextually accurate code.

• Open source library to scrape PDFs, YouTube, URLs, Presentations, etc for API-hosted vision-language models
  The open source library thepi.pe enables extraction of text and visual data from diverse sources like PDFs, webpages, and YouTube videos, specifically designed for seamless integration with API-hosted vision language models such as GPT-4-Vision or Gemini.