# Jan 2, 2025

- **Putnam-AXIOM** introduces a **new benchmark** for evaluating **higher-level mathematical reasoning** in large language models (LLMs), highlighting significant performance gaps and the effects of data contamination.

- **Deepseek v3** is a **607B mixture-of-experts model** with **37B active parameters**, outperforming both open-source and closed models like Llama 3.1 and Qwen, while achieving this with a training cost of only **$6 million**.

- **AI is revolutionizing the study of ancient texts**, enabling researchers to read previously inaccessible writings, such as the charred Herculaneum scrolls, which contain significant historical insights lost for 2,000 years.

- **TinyStories** demonstrates that language models with **fewer than 10 million parameters** can generate coherent and grammatically correct text, challenging the notion that larger models are necessary for fluency.

- The **RT-2 model** demonstrates advanced capabilities in **vision-language-action** tasks, successfully executing commands like "pick up the extinct animal" through integrated perception and action.

- **Austin Z. Henley's 25-year journey in AI and ML** began unexpectedly, evolving from simple programming projects to complex applications, highlighting the organic growth of his expertise in the field.

- **FineCodeX** is a platform that requires **JavaScript** to function, indicating its reliance on modern web technologies for optimal performance.

- The paper, "[Function Basis Encoding of Numerical Features in Factorization Machines](https://openreview.net/forum?id=M4222IBHsh)," introduces a novel approach to enhance **Factorization Machines (FMs)** by employing **parametric curves** for numerical feature encoding, improving their effectiveness in recommender systems.

- **Multihead latent attention** compresses the entire sequence into a **latent vector**, allowing the model to access information from the whole sequence, which disrupts traditional autoregressive constraints.

- **OpenAI o1** achieves expert-level performance through **reinforcement learning**, emphasizing the importance of **policy initialization**, **reward design**, **search**, and **learning** as critical components for replicating its capabilities.