# Sep 2, 2024

## Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer

- The **Fully Pipelined Distributed Transformer (FPDT)** significantly **increases the sequence length** capability for training Large Language Models (LLMs) on **limited hardware**, addressing the high resource demands of processing long contexts.

## Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs

- **LLMs demonstrate remarkable inductive reasoning capabilities** when isolated from deductive reasoning tasks, as shown by the novel **SolverLearner framework** which focuses on pure inductive reasoning by learning mappings from input to output.

## AI-Implanted False Memories

- **Conversational AI, particularly generative chatbots using large language models (LLMs), significantly amplifies the formation of false memories** in individuals during simulated crime witness interviews, compared to control, survey-based, and pre-scripted chatbot interactions.

## Population Minimizer of The Categorical Cross Entropy Loss (a blog post)

- The **population minimizer of conditional risk** associated with **Categorical Cross Entropy** is proven to be the **true probability label distribution**, a significant insight for neural network outputs.

## Fine-tuning coding LLMs on Git histories rather than just final code?

- **Fine-tuning coding LLMs on Git histories** could enhance software development by leveraging the **evolutionary context** of code, beyond its final state.

## MemLong: Memory-Augmented Retrieval for Long Text Modeling

- **MemLong**, a **Memory-Augmented Retrieval** method for **Long Text Generation**, addresses the challenge of handling long contexts in Large Language Models (LLMs) by utilizing an external retriever for historical information retrieval.

## Investigating Neuron Ablation in Attention Heads: The Case for Peak Activation Centering

- **Neuron ablation** in **transformer-based models** reveals **varying impacts** on performance, highlighting the **complexity of attention mechanisms**.

## A Unified Benchmark for Federated Unsupervised Anomaly Detection in Tabular Data

- The **FedAD-Bench** benchmark, introduced in a recent study, aims to **standardize the evaluation of unsupervised anomaly detection** in federated learning (FL) environments, specifically for tabular data. [Read the paper](https://arxiv.org/abs/2408.04442)

## Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning

- The **MVP framework** reduces hallucinations in Large Vision-Language Models ( **LVLMs**) without the need for retraining, leveraging **Multi-View Multi-Path Reasoning** to enhance comprehension of image content.

## EMPOWER: Embodied Multi-role Open-vocabulary Planning with Online Grounding and Execution

- **EMPOWER** introduces an **open-vocabulary online grounding and planning framework** for robots, enhancing their ability to perform tasks in real-life settings by addressing common challenges in task planning.

## Dynamic Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling

- **Reasoning-Aware Self-Consistency (RASC)** introduces an **early-stopping framework** that evaluates both the **output answer and the reasoning paths** to **optimize sample usage** and **enhance answer reliability** in Large Language Models.

## Safety Layers of Aligned Large Language Models: The Key to LLM Security

- **Aligned Large Language Models (LLMs)** are designed to be secure by **refusing to answer malicious questions**, but their **security mechanisms**, particularly at the **parameter level**, are not fully understood, especially under conditions of fine-tuning with non-malicious data.

## UserSumBench: A Benchmark Framework for Evaluating User Summarization Approaches

- **UserSumBench** is a **benchmark framework** designed to **overcome challenges** in developing LLM-based user summarization techniques, such as the lack of ground-truth labels and the costly nature of human evaluation.