Novel recurrent architectures like S4, Mamba, and Aaren have emerged due to the scalability limitations of Transformers, prompting a reevaluation of traditional RNNs such as LSTMs and GRUs.
Serving 70B-scale LLMs efficiently on low-resource edge devices
TPI-LLM presents a compute- and memory-efficient tensor parallel inference system that enables the deployment of 70B-scale models on low-resource edge devices, addressing privacy concerns by keeping sensitive data local.
Image Editing with Gaussian Splatting
MiraGe employs Gaussian Splatting to transform 2D images into a manipulable 3D space, allowing users to intuitively edit images by interpreting selections as geometric representations, enhancing flexibility in image editing.
FLUX1.1 [pro] – New SotA text-to-image model from Black Forest Labs
FLUX1.1 [pro] achieves six times faster generation than its predecessor, enhancing image quality, prompt adherence, and diversity for superior performance in text-to-image tasks.
[R] Were RNNs All We Needed?
The authors, including Y. Bengio, introduce simplified LSTM and GRU architectures that enable parallel training, yielding impressive benchmark results, as detailed in their arXiv paper.
Apple Depth Pro: Sharp monocular metric depth in less than a second
Depth Pro achieves zero-shot metric monocular depth estimation, producing high-resolution depth maps in 0.3 seconds on a standard GPU, without needing camera metadata.
[R] Announcing the first series of Liquid Foundation Models (LFMs)
Liquid Foundation Models (LFMs) represent a breakthrough in generative AI, achieving state-of-the-art performance across various scales (1B, 3B, and 40B parameters) while being memory efficient and optimized for long-context tasks, with a 32k token context length confirmed by the RULER benchmark.
[P] Larger and More Instructable Language Models Become Less Reliable
Larger models trained with extensive resources and human feedback exhibit increased unreliability, struggling with simple tasks while occasionally excelling at complex ones, leading to a paradox of performance inconsistency.
Update on Reflection-70B
Reflection 70B, a fine-tuned model based on Llama 3.1, has shown state-of-the-art benchmark scores but faced reproducibility issues due to initial miscommunication and methodological discrepancies.
Improving Accessibility Using Vision Models
Vision models, particularly Gemini 1.5 Flash, significantly enhance accessibility in math education by converting image-based equations into LaTeX, outperforming GPT-4o in accuracy and cost-effectiveness.
NumPy QuadDType: Quadruple Precision for Everyone
Numpy-QuadDType introduces a cross-platform quadruple precision datatype for NumPy, addressing the limitations of np.longdouble and ensuring consistent high precision across various architectures.
Show HN: open source framework OpenAI uses for Advanced Voice
OpenAI Realtime API integration in the Agents framework enables ultra-low latency communication between GPT-4o and user devices, enhancing real-time AI interactions.
[D] Which papers describe techniques for MOE in LLMs?
Key techniques for MOE in LLMs include methods from notable papers such as the Sparsely-Gated Mixture-of-Experts Layer and OLMoE, which enhance model efficiency and scalability.
General Preference Modeling with Preference Representations for Aligning Language Models
Preference representation learning introduces a novel method for efficiently capturing complex human preferences in a latent space, achieving linear query complexity compared to traditional models.