Recent AI Developments

• Deep-TEMPEST tackles the challenge of eavesdropping on HDMI by using deep learning to interpret electromagnetic waves, a significant leap from the unclear results of analog systems.

• Meta's SAM 2 model extends the capabilities of its predecessor by introducing a unified approach to segment objects in both images and videos, using a variety of inputs like clicks, boxes, or masks.

• Activision has released the Caldera map from Call of Duty: Warzone as an open-source data set, aiming to foster innovation and learning in AI and game development through academic and research communities.

• Meta's FAIR has launched SAM 2, an enhanced model capable of segmenting objects not only in images but now in video as well, maintaining its open-access ethos.

• The Ampere AmpereOne Aurora is a 512-core AI CPU designed for cloud-native AI computing, marking a significant advancement in processor technology.

• Gemma Scope is a new suite of tools designed to enhance the interpretability of language models, particularly focusing on Gemma 2's inner mechanisms through hundreds of open sparse autoencoders.

• Amanda Bertsch discusses the "Unlimiformer" architecture, aiming for unlimited context length in NLP, at the Oxen.ai Paper Club this Friday. Oxen Community Call

• ThinK introduces a query-dependent KV cache pruning method that targets the channel dimension's redundancy, effectively reducing memory costs without sacrificing model accuracy.

• Neurosymbolic AI combines symbolic reasoning and deep learning to enhance reasoning over knowledge graphs, which represent complex, multi-relational data.

• JaColBERTv2.5 significantly advances Japanese neural information retrieval by optimizing multi-vector retrievers for constrained resources, outperforming multilingual models in capturing linguistic nuances.

• SAM2 outperforms popular object tracking frameworks like bytetrack, botsort, and deepsort in tests, without the need for fine-tuning or parameter adjustments.

• The Annotation Vocabulary introduces a transformer-readable language for proteins, focusing on biochemically relevant properties without relying on amino acid sequences, thereby creating a new dimension for protein embeddings. Read the paper

• CLEFT introduces an efficient language-image contrastive learning method, leveraging large pre-trained models and fine-tuning prompts to bridge the gap between complex clinical data and simple class labels.

• Torchchat is a new library by PyTorch that enables efficient running of Llama 3, 3.1, and other large language models (LLMs) on various platforms including laptops, desktops, and mobile devices, expanding upon previous work with native PyTorch 2.0 and CUDA for enhanced performance across more environments.

• OmniBal significantly reduces the computational load imbalance in vision-language instruct-tuning models by addressing data distribution and model architecture heterogeneity.