# AI Innovations Recap

- **Interesting Interview with DeepSeek's CEO**
  - **Deepseek's R1 model has outperformed OpenAI’s o1 on reasoning benchmarks**, showcasing its potential as a formidable player in the AI landscape, driven by a commitment to foundational technology and open-source principles.

- **Things we learned about LLMs in 2024**
  - **In 2024, the GPT-4 barrier was decisively surpassed**, with 18 organizations now boasting models that outperform it, including Google's Gemini 1.5 Pro, which introduced unprecedented capabilities like a 2 million token input context and video input.

- **Coconut by Meta AI – Better LLM Reasoning with Chain of Continuous Thought?**
  - **Coconut** introduces a novel reasoning method for LLMs, allowing them to operate in a **continuous latent space**, enhancing their problem-solving capabilities beyond traditional word-based reasoning.

- **Beyond Gradient Averaging in Parallel Optimization**
  - **Gradient Agreement Filtering (GAF)** enhances distributed deep learning by filtering out conflicting updates, leading to improved **validation accuracy** and reduced memorization of noisy labels.

- **How Well Do LLMs Generate Code for Different Application Domains?**
  - **MultiCodeBench** introduces a comprehensive benchmark with **2,400 programming tasks** across **12 application domains** and **15 programming languages**, addressing the gap in existing code generation evaluations for specific contexts.

- **New SOTA Text to Audio model using rectified flow and FLUX architecture**
  - A **new state-of-the-art (SOTA)** Text to Audio model utilizes **rectified flow matching** and **FLUX architecture**, achieving rapid inference times of about **3 seconds on a GPU**.

- **Introducing LongTalk-CoT v0.1: A Very Long Chain-of-Thought Dataset for Reasoning Model Post-Training**
  - **LongTalk-CoT v0.1** is a **post-training dataset** featuring **97M tokens**, specifically crafted to enhance reasoning models through **vocalised thinking** and **self-reflection** prompts.

- **What's the current SOTA for Biomedical Encoder Models?**
  - The current **SOTA** for biomedical encoder models includes **PubMedBERT** and **BioBERT**, which serve as strong baselines for tasks like **sentence similarity** and **document retrieval**.

- **Best approach for Object Detection with Absolute Scale without using a reference**
  - **CNN with regression heads** is proposed for predicting **absolute clearance distances** without reference objects, addressing the challenge of varying object dimensions and camera intrinsics.

- **Research Galore From 2024: Recapping AI Advancements in 3D Simulation, Climate Science and Audio Engineering**
  - **NVIDIA Research** has made significant strides in **generative AI**, enabling the creation of **3D models, music, and realistic humanoid motion**, enhancing both creative and scientific applications.

- **Dancing on Their Own: NVIDIA-Powered Robots to Adore From 2024**
  - **NVIDIA-powered robots** are set to revolutionize various sectors in 2024, with innovations like Clearbot's autonomous trash-collection boats and Figure's humanoid robots enhancing environmental and manufacturing efforts.

- **What kind of learning this is? What are different training paradigms for this type of learning?**
  - The proposed **stacked model** integrates a **Physics Informed Neural Network (PINN)** with a **Temporal Convolution Network (TCN)**, allowing for a **dynamic exchange of losses** between the two models during backpropagation, enhancing learning efficiency.

- **KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation**
  - **KARPA** introduces a **training-free** method for adapting knowledge graphs, enhancing large language models' reasoning capabilities without the need for fine-tuning or pre-training on specific datasets.