# Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models

**Procedural knowledge** in pretraining significantly enhances reasoning capabilities in Large Language Models (LLMs), revealing that models utilize distinct data sets for factual versus reasoning tasks, with procedural documents being crucial for the latter.

---

# DynaSaur: Large Language Agents Beyond Predefined Actions

**DynaSaur** introduces a novel framework for LLM agents that enables **dynamic action creation** and composition, overcoming limitations of fixed action sets in complex environments.

---

# Large Language Models as Markov Chains

**Large language models (LLMs)** can be equivalently represented as **Markov chains**, revealing insights into their **inference power** and **convergence speed** based on vocabulary size and context window.

---

# Controlling AI's Growing Energy Needs

**AI training consumes vast energy**, with models like ChatGPT-3 using nearly **1,300 megawatt hours**, equivalent to the annual energy use of **130 homes**; this exponential growth in energy demand poses sustainability challenges.

---

# How We Optimize LLM Inference for AI Coding Assistant

**Full codebase context is essential for developer AI**, as it significantly enhances the quality of code predictions, with Augment achieving a time to first token (TTFT) of less than **300ms** for 10k input tokens, outperforming existing solutions by threefold.

---

# We need data engineering benchmarks for LLMs

**Data engineering benchmarks are essential** for evaluating LLMs, as current frameworks like SWE-bench fail to address the unique challenges of data workflows, which prioritize **data quality, reliability, and scalability** over mere code correctness.

---

# [P] TIME-MOE: Billion-Scale Time Series Forecasting with Mixture-of-Experts

**Time-MOE** is a **2.4B parameter** open-source model that leverages **Mixture-of-Experts (MOE)** for effective **zero-shot forecasting** in time series analysis.