Language models are Super Mario: Absorbing abilities from homologous models
Language Models (LMs) can now absorb new abilities from similar models through a novel technique called DARE, which simplifies the merging of capabilities without the need for retraining or advanced hardware.
Long-form factuality in large language models
Large language models (LLMs) often produce content with factual inaccuracies in response to open-ended, fact-seeking prompts, prompting the development of LongFact, a benchmark using GPT-4 across 38 topics to assess long-form factuality.
AI and the Problem of Knowledge Collapse
AI's potential for vast data processing and productivity gains is counterbalanced by the risk of knowledge collapse, a phenomenon where reliance on AI narrows public understanding and innovation.
Fortran on WebAssembly
Dr. George W Stagg demonstrates compiling Fortran code to WebAssembly using a patched LLVM's flang-new compiler, enabling Fortran routines to run in web browsers.
Autonomous Overhead Powerline Recharging for Uninterrupted Drone Operations
The research introduces a drone capable of autonomously recharging from overhead power lines, enabling sustained, uninterrupted operations over multiple hours by leveraging an onboard perception and autonomy component for power line detection and navigation.
ML researchers who are not in NLP, what are you researching? Please share.
ML researchers outside the NLP domain are invited to share their research areas, aiming for a comprehensive view of the spectrum of ML research.
Loki: An open-source tool for fact verification
Loki is an open-source tool designed to automate fact verification, providing a pipeline that includes decomposing texts into claims, assessing their significance, generating queries, crawling for evidence, and verifying claims, aimed at journalists and researchers.
Chisel: A fast TCP/UDP tunnel over HTTP
Chisel is a fast TCP/UDP tunnel over HTTP, secured with SSH, combining both client and server functionalities into a single Go executable, designed for firewall traversal and secure network entry.
How do you get enough patience to train-debug-train models?
Patience in model training is tested when errors emerge after long periods of waiting, highlighting the frustration of paused tasks and wasted time.
Tokens, N-Grams, and Bag-of-Words Models
Tokens and n-grams serve as foundational elements in Natural Language Processing (NLP), enabling the understanding and manipulation of text by breaking it down into manageable units for analysis.
PID Controller Explainer (2022)
PID controllers adjust system inputs based on the difference between a desired set point and an actual sensor measurement, optimizing for minimal error without requiring knowledge of the system's internal mechanics.
Schedule-Free Learning – A New Way to Train
Schedule-Free learning employs a novel approach by replacing the momentum of an underlying optimizer with interpolation and averaging, facilitating faster training without the need for predefined schedules.
Optuna – A Hyperparameter Optimization Framework
Optuna is an open-source framework designed for the automation of hyperparameter optimization, featuring eager search spaces, state-of-the-art algorithms, and easy parallelization.
I just can't fine tune BERT over 40% accuracy for text-classification task
The initial struggle to fine-tune BERT for a text-classification task with 40% accuracy was overcome by addressing data insufficiency and class imbalance, leading to an 88.45% accuracy on the validation set.
Photoshop for Text (2022)
Text editing is evolving beyond basic functions like cut and paste to include complex transformations, such as altering style or summarizing content, akin to image manipulation in Photoshop.