May 29, 2024

Daily Updates

• Reproducing GPT-2 (124M) using llm.c on a single 8X A100 80GB SXM node takes approximately 90 minutes and costs around $20, showcasing an efficient use of model flops utilization at up to ~60%.

• Codestral, Mistral AI's inaugural code model, excels in code generation across 80+ programming languages, promising to enhance software development with its advanced AI capabilities.

• The world's first bioprocessor, developed by Swiss startup FinalSpark, utilizes 16 human brain organoids to achieve ‘a million times less power’ consumption than traditional digital chips.

• Llama 3-V is a multimodal model that matches GPT4-V's performance with a 100x smaller model size, achieving a 10-20% performance boost over Llava, the current state-of-the-art (SOTA) in multimodal understanding, by leveraging a novel architecture and training approach.

• Over the past year, building with large language models (LLMs) has transitioned from experimental to practical, with real-world applications benefiting from the technology's advancements and broad accessibility.

• tinygrad 0.9.0 introduces significant usability improvements, including over 1200 commits since version 0.8.0, and approaches the new line limit with 7958 lines of code.

• AdFlush, a machine learning model, was developed for real-world browsers to effectively prevent advertisements and web trackers, selecting 27 key features from an evaluation of 883 for optimal performance.

• Cohere's release of the Wikipedia dataset, embedded into vectors using their multilingual-v3 model, makes it feasible to index Wikipedia on a personal laptop, sidestepping the previously prohibitive $5000 cost of computing such embeddings. Cohere's dataset

• ChatTTS is a text-to-speech model tailored for dialogue scenarios, supporting English and Chinese with a training dataset of over 100,000 hours, while the open-source version is based on 40,000 hours of pre-training without Speaker Feature Transfer (SFT).

• Hallucination in LLMs is perceived as less prioritized than safety, despite its critical role in ensuring the accuracy of generated information.

• Elixir's machine learning landscape has evolved with Nx v0.7 introducing MLIR support, enhancing capabilities like Apple Silicon Metal support and cross-compilation for embedded devices. MLIR

• Optimised Attention reduces parameters by 25% and matrix multiplications per head, maintaining performance akin to standard attention.

• Era3D introduces a novel multiview diffusion method that overcomes camera prior mismatch and inefficacy, generating high-resolution images from a single image without shape distortions.

• REX Computing is innovating with a new processor architecture, the Neo, aiming for a 10 to 25x increase in energy efficiency over current CPUs and GPUs by simplifying design and focusing on software improvements.

• The You Only Cache Once (YOCO) architecture introduces a novel split in the network into self-decoder layers for generating a global KV-Cache and cross-decoder layers that reuse this cache, aiming to enhance efficiency in Large Language Models (LLMs) as illustrated in Figure 1.