The Lambda Deep Learning Blog

Tesla A100 Server Total Cost of Ownership Analysis

This post compares the Total Cost of Ownership (TCO) for Lambda servers and clusters vs cloud instances with NVIDIA A100 GPUs. We first calculate the TCO for ...

RTX A6000 vs RTX 3090 Deep Learning Benchmarks

Check out the discussion on Reddit 160 upvotes, 41 comments

OpenAI's GPT-3 Language Model: A Technical Overview

by Chuan Li, PhD

Training Neural Networks in Record Time with the Hyperplane-16

by Chuan Li, PhD

TensorFlow 2.0 Tutorial 01: Basic Image Classification

TensorFlow 2 is now live! This tutorial walks you through the process of building a simple CIFAR-10 image classifier using deep learning. In this tutorial, we ...

Setting up Horovod + Keras for Multi-GPU training

This blog will walk you through the steps of setting up a Horovod + Keras environment for multi-GPU training.

Tracking system resource (GPU, CPU, etc.) utilization during training with the Weights & Biases Dashboard

One of the most asked questions we get at Lambda Labs is, “how do I track resource utilization for deep learning jobs?” Resource utilization tracking can help ...

TensorFlow 2.0 Tutorial 05: Distributed Training across Multiple Nodes

Distributed training allows scaling up deep learning task so bigger models can be learned or training can be conducted at a faster pace. In a previous ...

TensorFlow 2.0 Tutorial 04: Early Stopping

During training, weights in the neural networks are updated so that the model performs better on the training data. For a while, improvements on the training ...

TensorFlow 2.0 Tutorial 03: Saving Checkpoints

This tutorial combines two items from previous tutorials: saving models and callbacks. Checkpoints are saved model states that occur during training. With ...

TensorFlow 2.0 Tutorial 02: Transfer Learning

This tutorial shows you how to perform transfer learning using TensorFlow 2.0. We will cover:

V100 server on-prem vs AWS p3 instance cost comparison

Deep Learning requires GPUs, which are very expensive to rent in the cloud. In this post, we compare the cost of buying vs. renting a cloud GPU server. We use ...

Text Generation: Char-RNN Data preparation and TensorFlow implementation

This tutorial is about making a character-based text generator using a simple two-layer LSTM. It will walk you through the data preparation and the network ...

Multi-GPU enabled BERT using Horovod

BERT is Google's pre-training language representations which obtained the state-of-the-art results on a wide range of Natural Language Processing tasks. ...

Reproduce Fast.ai/DIUx imagenet18 with a Titan RTX server

Last, year, Fast.ai won the first ImageNet training cost challenge as part of the DAWN benchmark. Their customized ResNet50 takes 3.27 hours to reach 93% ...