The 2024 Nobel Prize in Physiology or Medicine was awarded to Victor Ambros and Gary Ruvkun for their groundbreaking work on microRNA and its critical function in post-transcriptional gene regulation.
Longwriter – Increase llama3.1 output to 10k words
LongWriter is a groundbreaking model capable of generating over 10,000 words in just one minute, utilizing advanced long-context LLMs, and is now deployable via vllm for enhanced performance.
ByteDance’s Bytespider is scraping at much higher rates than other platforms
ByteDance's Bytespider is a web scraper that operates 25 times faster than OpenAI's GPTbot, rapidly accumulating data to enhance its generative AI models, reflecting a strategic push to catch up in the AI race.
The magic (image resampling) kernel
Magic Kernel Sharp is a superior image resizing algorithm, utilized by major platforms like Facebook and Instagram, enhancing image quality while improving CPU and storage efficiency since its inception in 2013.
Sorbet: A neuromorphic hardware-compatible transformer-based spiking model
Sorbet introduces a neuromorphic hardware-compatible transformer-based spiking language model that utilizes PTsoftmax and BSPN to replace energy-intensive operations, enhancing efficiency for edge deployment.
[P] Model2Vec: Distill a Small Fast Model from any Sentence Transformer
Model2Vec distills Sentence Transformer models into 30mb static embeddings that are up to 500x faster than their original counterparts, enabling efficient CPU usage without requiring extensive hardware.
Llamafile for Meltemi: The First LLM for Greek
Meltemi 7B Instruct v1.5 is the first Large Language Model (LLM) for Greek, developed by the Athena Research & Innovation Center, and is available in both llamafile and gguf formats on HuggingFace.
[R] MaskBit: Embedding-free Image Generation via Bit Tokens
MaskBit introduces an innovative, embedding-free image generation model that operates directly on bit tokens, achieving a state-of-the-art FID of 1.52 on the ImageNet 256x256 benchmark with a compact generator of 305M parameters.
[Project] Optimizing Neural Networks with Language Models
Dux is a meta-optimizer leveraging GPT-4o-mini for the adaptive optimization of neural networks, aiming to enhance performance through innovative techniques.
Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models
Object hallucinations in CLIP models are a significant concern, revealing that these issues arise independently of the interaction between vision and language modalities, indicating a deeper flaw within the model itself.
MELODI: Exploring Memory Compression for Long Contexts
MELODI introduces a hierarchical compression scheme that optimally balances short-term and long-term memory, enabling efficient processing of long documents with limited context windows.
LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
LoRC introduces a low-rank approximation for KV weight matrices, enabling significant memory reduction in transformer-based LLMs without the need for model retraining or extensive parameter tuning.