# Jul 9, 2024

- **MobileLLM:** Optimizing sub-billion parameter language models for on-device use, integrating SwiGLU activation, deep and thin architectures, embedding sharing, and grouped-query attention, achieving **up to 4.3% accuracy improvement** over state-of-the-art models on zero-shot commonsense reasoning tasks. [Read the paper](https://arxiv.org/abs/2402.14905)

- **Surprising Gender Biases in GPT:** The **OSF (Open Science Framework)** platform has implemented **Google Tag Manager** and **Sentry** for enhanced tracking and error logging, respectively, to improve site functionality and user experience.

- **turbopuffer:** A **cost-efficient, high-performance search engine** that leverages **object storage and smart caching**, designed to scale to billions of vectors and millions of tenants/namespaces, addressing the high costs and operational challenges of existing search solutions.

- **[R] Learning to (Learn at Test Time):** RNNs with Expressive Hidden States. The **Test-Time Training (TTT) layers** introduce a novel approach by making the hidden state a **machine learning model itself**, enhancing expressiveness and performance in long contexts.

- **LightRAG:** The PyTorch Library for Large Language Model Applications. **LightRAG** is designed to assist developers in creating and optimizing **Retriever-Agent-Generator (RAG) pipelines** for a wide range of applications, from chatbots to text classification, emphasizing its **lightweight, modular, and robust architecture**.

- **LLaMAX:** Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages. **LLaMAX enhances translation capabilities** of Large Language Models (LLMs) to support over **100 languages**, focusing on improving performance in low-resource languages through extensive multilingual continual pre-training.

- **[R] What is GraphRAG? Explained:** **GraphRAG** represents an **advancement over the baseline RAG** by utilizing **Knowledge Graphs** for retrieval, which enhances the **quality of its output**.

- **[D] Inference-time gradients:** **Diffusion models** excel in **inference-time gradients** by predicting **score/noise/gradient** to reverse the diffusion process, notably through **[rectified flow](https://arxiv.org/abs/2209.03003)** in **[Stable Diffusion 3](https://arxiv.org/abs/2403.03206)**.

- **[Research] Neural decoding - mapping EEG data of song listening to respective audio files:** The **research aims to reconstruct songs** participants listened to by mapping **EEG data to audio files** using a regression model, with a **CNN-based approach** detailed in a [study](https://arxiv.org/abs/2207.13845).

- **Multi-Object Hallucination in Vision-Language Models:** **Large vision language models (LVLMs) exhibit increased object hallucination when processing multiple objects**, revealing a significant challenge in accurately recognizing and reasoning about complex visual scenes.

- **[R] Literature on Lipsync/Body Gestures:** The inquiry seeks **research papers on body gesture and lipsync training**, aiming to understand the methodologies behind technologies like **Heygen and Synthesia**.

- **[R] How GraphRAG works? Explained:** **GraphRAG** enhances **RAG (Retrieval-Augmented Generation)** by improving the connection of external documents to **Large Language Models (LLMs)**, facilitating more informed and context-rich responses.

- **TwoMinutePapers - DeepMind’s New AI Found The Sound Of Pixels!** **DeepMind's new AI technique** synthesizes sound by analyzing video content, a significant leap towards more immersive AI-generated media.

- **In It for the Long Haul:** **Waabi** is leveraging **generative AI** and **NVIDIA's DRIVE Thor technology** to revolutionize autonomous trucking, aiming for **fully driverless operations** by next year.

- **YannicKilcher - Scalable MatMul-free Language Modeling (Paper Explained):** Researchers from **UC Santa Cruz, Sucha University, UC Davis, and Loxy Tech** have developed a **scalable, MatMul-free language model** that replaces matrix multiplication in large language models with ternary accumulators and a parallelizable form of a ternary recurrent network, aiming for **greater hardware efficiency**.