Jun 19, 2024

• Safe Superintelligence Inc. (SSI) aims to develop the world's first safe superintelligence, addressing both safety and capabilities through revolutionary engineering.

• Meta FAIR has released six new research artifacts focusing on innovation, creativity, efficiency, and responsibility, including models for image-to-text and text-to-music generation, and a technique for detecting AI-generated speech.

• Refusal behavior in conversational large language models, which are designed to reject harmful instructions, is controlled by a single direction in their operational mechanics, as discovered across 13 popular models.

• YaFSDP is a Sharded Data Parallelism framework optimized for transformer-like neural networks, offering detailed insights through Medium and Habr blog posts.

• The 3D Gaussian Splatting method for neural rendering is enhanced by treating the set of 3D Gaussians as Markov Chain Monte Carlo (MCMC) samples, leading to higher quality scene reconstructions without the need for precise initial placements.

• Large language models (LLMs) rely on complex data pipelines for dataset creation, with Common Crawl's WARC/WAT/WET formats serving as a primary data source, highlighting the importance of data quality and preprocessing in model training. Common Crawl

• Slack's engineering team successfully automated the conversion of 15,000 unit and integration tests from Enzyme to React Testing Library, achieving an 80% success rate by integrating Abstract Syntax Trees (AST) with Large Language Models (LLMs), specifically using Anthropic's Claude 2.1.

• Early efforts to enhance code completion for Rust in Cody have shown promising results, particularly in addressing the performance gap observed in languages not well-represented in training datasets.

• The paper by Liesenfeld, Lopez, and Dingemanse (2023) critically examines the openness of instruction-tuned text generators like ChatGPT, highlighting the importance of open research for scientific progress and informed decision-making in AI deployment.

• Evolutionary Strategy (ES) for training neural networks achieves 90% accuracy without using gradient information, matching the speed of backpropagation on GPUs.

• Ilya Sutskever, a prominent figure in AI, has co-founded Safe Superintelligence Inc. with a focus on developing Artificial Superintelligence (ASI) without the distraction of product cycles.

• Logit Prisms extend the logit lens method by mathematically decomposing transformer outputs into contributions from individual components like attention heads and MLP neurons, revealing how each part influences the final decision.

• The VkFFT library creator benchmarked AMD MI300X and Nvidia H100 GPUs in FFT tasks, revealing that both GPUs deliver similar performance in single precision, with bandwidths around 3TB/s, but fall short of their theoretical maximums. VkFFT

• DiTTo-TTS introduces an efficient and scalable Diffusion Transformer for Text-to-Speech, leveraging pre-trained text and speech encoders to overcome traditional text-speech alignment challenges.

• Ilya Sutskever, co-founder and former chief scientist of OpenAI, is launching Safe Superintelligence Inc. (SSI), an AI startup dedicated to developing a safe and powerful AI system, emphasizing safety alongside capabilities.