# Oct 28, 2024

## High-resolution postmortem human brain MRI at 7 tesla
- **Ex vivo MRI** at **7 tesla** offers superior visualization of neuroanatomy, enabling the integration of microscale histology with morphometric analysis, which is crucial for advancing brain research.

## State-space models can learn in-context by gradient descent
- **Deep state-space models (Deep SSMs)** can achieve in-context learning through **gradient descent**, demonstrating that a single structured layer with local self-attention can replicate outputs of an implicit linear model after just one gradient descent step.

## A return to hand-written notes by learning to read and write
- **A novel model converts handwriting photos into digital ink**, capturing stroke-level details without specialized equipment, enhancing the natural feel of handwritten notes in digital form.

## Demystifying distributed checkpointing
- **Distributed checkpointing** is essential for optimizing **LLM training workflows**, allowing for efficient recovery from failures and minimizing wasted compute resources during lengthy training processes.

## New Interview with Leland McInnes: UMAP, HDBSCAN & the Geometry of Data | Learning from Machine Learning #10

## Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
- **Diffusion models** offer a promising alternative to **autoregressive generation** for complex reasoning and planning tasks, potentially enhancing performance in compositional domains like math and logic.

## Last Week in Medical AI: Top LLM Research Papers/Models (October 19 - October 26)
- **Google's paper on safety principles for medical summarization highlights the transformative potential of generative AI in healthcare workflows, addressing both its promise and inherent challenges.**

## Two are better than one: Context window extension with multi-grained self-injection
- **SharedLLM** enhances large language models (LLMs) by utilizing a **multi-grained context compression** approach, allowing for efficient context window extension without the high costs of continual pre-training on long-context data.

## Bring Receipts: New NVIDIA AI Workflow Detects Fraudulent Credit Card Transactions
- **NVIDIA's new AI workflow on AWS enhances fraud detection** by utilizing accelerated data processing and advanced algorithms, significantly improving accuracy and reducing false positives in credit card transactions.

## Expert Support case study: Bolstering a RAG app with LLM-as-a-Judge
- **Farmer.chat** utilizes a **Retrieval-Augmented Generation (RAG)** pipeline powered by LLMs to deliver personalized agricultural advice, effectively addressing the needs of over **20,000 farmers** and handling **340,000 queries** in multiple languages.

## BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
- **BitPipe** introduces a **bidirectional interleaved pipeline parallelism** that significantly enhances training efficiency for large models by reducing computational time and increasing simultaneous device usage.

## Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning
- **HeadKV** introduces a **head-level KV cache compression method** that selectively retains critical information, achieving **97% performance** of the full cache while using only **1.5%** of its size on contextual QA tasks.

## TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning
- **TimeSuite** enhances **Multimodal Large Language Models (MLLMs)** for long video understanding by introducing a novel framework, a high-quality dataset, and a new instruction tuning task that incorporates temporal grounding, significantly improving performance metrics.

## Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs
- **XGBLoRA** introduces a novel framework that enhances **Low-Rank Adaptation** (LoRA) for fine-tuning **Large Language Models** (LLMs) by utilizing ensemble learning to improve model predictions while maintaining computational efficiency.

## Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models
- **Prompt engineering** is crucial in mitigating **hallucinations** in Large Language Models (LLMs), with empirical evaluations revealing that simpler prompting techniques often yield better results than complex ones.