Jan 2, 2025

• Putnam-AXIOM introduces a new benchmark for evaluating higher-level mathematical reasoning in large language models (LLMs), highlighting significant performance gaps and the effects of data contamination.

• Deepseek v3 is a 607B mixture-of-experts model with 37B active parameters, outperforming both open-source and closed models like Llama 3.1 and Qwen, while achieving this with a training cost of only $6 million.

• AI is revolutionizing the study of ancient texts, enabling researchers to read previously inaccessible writings, such as the charred Herculaneum scrolls, which contain significant historical insights lost for 2,000 years.

• TinyStories demonstrates that language models with fewer than 10 million parameters can generate coherent and grammatically correct text, challenging the notion that larger models are necessary for fluency.

• The RT-2 model demonstrates advanced capabilities in vision-language-action tasks, successfully executing commands like "pick up the extinct animal" through integrated perception and action.

• Austin Z. Henley's 25-year journey in AI and ML began unexpectedly, evolving from simple programming projects to complex applications, highlighting the organic growth of his expertise in the field.

• FineCodeX is a platform that requires JavaScript to function, indicating its reliance on modern web technologies for optimal performance.

• The paper, "Function Basis Encoding of Numerical Features in Factorization Machines," introduces a novel approach to enhance Factorization Machines (FMs) by employing parametric curves for numerical feature encoding, improving their effectiveness in recommender systems.

• Multihead latent attention compresses the entire sequence into a latent vector, allowing the model to access information from the whole sequence, which disrupts traditional autoregressive constraints.

• OpenAI o1 achieves expert-level performance through reinforcement learning, emphasizing the importance of policy initialization, reward design, search, and learning as critical components for replicating its capabilities.