Project Sid: Many-agent simulations toward AI civilization

Project Sid explores large-scale simulations of 10 to 1000+ AI agents, revealing their ability to develop specialized roles and engage in cultural transmission, thus paving the way for future AI civilizations.

Speed, scale and reliability: 25 years of Google datacenter networking evolution

Jupiter's evolution has culminated in a 13 Petabit per second network, capable of supporting video calls for all 8 billion people on Earth simultaneously, showcasing unprecedented scalability in data center networking.

Hertz-dev, the first open-source base model for conversational audio

hertz-dev is an open-sourced, full-duplex audio-only transformer model with 8.5 billion parameters, designed for advanced speech generation and interaction.

An embarrassingly simple approach to recover unlearned knowledge for LLMs

LLMs may not truly forget: This study reveals that applying quantization to unlearned models can restore up to 83% of the "forgotten" knowledge, challenging the effectiveness of current unlearning methods.

What is your Recipe for Training Neural Networks in 2024?

Modern training recipes for neural networks in 2024 must adapt to the evolving landscape of Deep Learning, where supervised learning remains more effective than unsupervised methods, particularly in image diffusion models.

Understanding Multimodal LLMs: The Main Techniques and Latest Models

Multimodal LLMs integrate various data types—text, images, audio, and video—enabling applications like image captioning and PDF data extraction, with Meta AI's Llama 3.2 models exemplifying this capability through open-weight versions for large language models.

Scalable watermarking for identifying large language model outputs

SynthID-Text is a novel watermarking scheme for large language models (LLMs) that ensures high-quality text generation while enabling effective detection of synthetic outputs, addressing the challenges of quality, detectability, and computational efficiency in production systems.

Hermes 3 – Nous Research

Hermes 3 introduces a new frontier model of open-source AI, featuring advanced long-term context retention, multi-turn conversation capabilities, and enhanced agentic function-calling, significantly improving reasoning and creativity compared to Llama 3.1.

Fourier weights neural networks

Fourier weights neural networks (FWNN) utilize Fourier coefficients to parametrize weights, allowing for a symmetric weight structure that can enhance learning efficiency compared to traditional MLPs.

Benchmarking 1 Million Files from ImageNet into DVC, Git-LFS, and Oxen.ai for Open Source Dataset Collaboration

Oxen.ai has benchmarked its open source unstructured data version control tool against 1 million+ images from the ImageNet dataset, revealing it to be 13x faster than git-lfs and 5x faster than DVC for data uploads.

Video Input for the current LLMs

OpenSceneSense is a Python package that enables large language models (LLMs) to analyze video content through frame-by-frame analysis, audio transcription, and scene detection, enhancing their capability to generate insights from video data.