# Jul 6, 2024

## License Update
- Stability AI introduces a **new Community License** allowing free use of its models for research, non-commercial, and commercial purposes, provided annual revenues do not exceed USD$1M.

## Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
- **Diffusion Forcing** introduces a **novel training paradigm** where a diffusion model denoises tokens with varied noise levels, merging next-token prediction's variable-length generation with full-sequence diffusion's trajectory guidance.

## New AI Training Technique Is Drastically Faster, Says Google
- Google has developed a **new AI training technique** that significantly **reduces the time** required for training AI models.

## Tokens are a big reason today's generative AI falls short
- **Generative AI models rely on tokenization**, breaking down text into smaller units like words or characters, to process language, which introduces biases and limitations.

## A rapid response system for elderly safety monitoring
- The **Rapid Response Elderly Safety Monitoring (RESAM)** system utilizes **progressive hierarchical action recognition** and **multi-sensor data fusion** to efficiently reduce processing time and lower the false-negative rate for elderly safety monitoring.

## Exponential Growth of Context Length in Language Models
- **Language models' context lengths** have seen **exponential growth**, evolving from **512 tokens** in early models like **T5/BERT/GPT-1** to **2 million tokens** in the latest **Gemini 1.5 Pro**.

## Releasing my loss function based on VGG Perceptual Loss
- The **VGG Loss** project enables the use of **pretrained models from PyTorch** as a base for training new models, inspired by the ability of VGG Loss to extract and transfer knowledge between models.

## Constrained decoding as stateful navigation?
- **Constraining a Large Language Model's (LLM's) decoder to specific grammars** like XML or JSON can enhance the reliability of a wrapping program's parsing process, but may be suboptimal without grammar-specific training.

## Is this true, sequential processing in O(log n)
- A student claims to have developed an **architecture** that outperforms **RNNs and transformers** in sequential processing, achieving **O(log n)** efficiency, and shows promise in **image generation**. [Read more](https://medium.com/@DakshishSingh/equinox-architecture-divide-and-compute-99c555ac08d6)

## Problems with Grad-CAM visualization for medical imaging
- **Grad-CAM visualizations** in medical imaging, specifically for **grayscale chest thermograms**, often **highlight irrelevant features** such as arms and shoulders, **overlooking the chest area** crucial for detecting abnormalities.