# Oct 4, 2024

## Were RNNs all we needed?
- **Novel recurrent architectures** like S4, Mamba, and Aaren have emerged due to the **scalability limitations** of Transformers, prompting a reevaluation of traditional RNNs such as LSTMs and GRUs.

## Serving 70B-scale LLMs efficiently on low-resource edge devices 
- **TPI-LLM** presents a **compute- and memory-efficient tensor parallel inference system** that enables the deployment of **70B-scale models** on low-resource edge devices, addressing privacy concerns by keeping sensitive data local.

## Image Editing with Gaussian Splatting
- **MiraGe** employs **Gaussian Splatting** to transform 2D images into a manipulable 3D space, allowing users to intuitively edit images by interpreting selections as geometric representations, enhancing flexibility in image editing.

## FLUX1.1 [pro] – New SotA text-to-image model from Black Forest Labs
- **FLUX1.1 [pro]** achieves **six times faster generation** than its predecessor, enhancing **image quality**, **prompt adherence**, and **diversity** for superior performance in text-to-image tasks.

## [R] Were RNNs All We Needed?
- The authors, including **Y. Bengio**, introduce **simplified LSTM and GRU** architectures that enable **parallel training**, yielding impressive benchmark results, as detailed in their [arXiv paper](https://arxiv.org/abs/2410.01201).

## Apple Depth Pro: Sharp monocular metric depth in less than a second
- **Depth Pro** achieves **zero-shot metric monocular depth estimation**, producing high-resolution depth maps in **0.3 seconds** on a standard GPU, without needing camera metadata.

## [R] Announcing the first series of Liquid Foundation Models (LFMs) 
- **Liquid Foundation Models (LFMs)** represent a breakthrough in generative AI, achieving **state-of-the-art performance** across various scales (1B, 3B, and 40B parameters) while being **memory efficient** and optimized for long-context tasks, with a **32k token context length** confirmed by the RULER benchmark.

## [P] Larger and More Instructable Language Models Become Less Reliable
- **Larger models** trained with extensive resources and human feedback exhibit **increased unreliability**, struggling with simple tasks while occasionally excelling at complex ones, leading to a paradox of performance inconsistency.

## Update on Reflection-70B
- **Reflection 70B**, a fine-tuned model based on Llama 3.1, has shown **state-of-the-art benchmark scores** but faced reproducibility issues due to initial miscommunication and methodological discrepancies.

## Improving Accessibility Using Vision Models
- **Vision models, particularly Gemini 1.5 Flash, significantly enhance accessibility in math education by converting image-based equations into LaTeX, outperforming GPT-4o in accuracy and cost-effectiveness.**

## NumPy QuadDType: Quadruple Precision for Everyone
- **Numpy-QuadDType** introduces a **cross-platform quadruple precision** datatype for NumPy, addressing the limitations of `np.longdouble` and ensuring consistent high precision across various architectures.

## Show HN: open source framework OpenAI uses for Advanced Voice
- **OpenAI Realtime API** integration in the **Agents framework** enables ultra-low latency communication between GPT-4o and user devices, enhancing real-time AI interactions.

## [D] Which papers describe techniques for MOE in LLMs?
- **Key techniques for MOE in LLMs** include methods from notable papers such as the **Sparsely-Gated Mixture-of-Experts Layer** and **OLMoE**, which enhance model efficiency and scalability.

## General Preference Modeling with Preference Representations for Aligning Language Models
- **Preference representation learning** introduces a novel method for efficiently capturing complex human preferences in a latent space, achieving **linear query complexity** compared to traditional models.