The Lambda Deep Learning Blog

Setting up Horovod + Keras for Multi-GPU training

This blog will walk you through the steps of setting up a Horovod + Keras environment for multi-GPU training.

Tracking system resource (GPU, CPU, etc.) utilization during training with the Weights & Biases Dashboard

One of the most asked questions we get at Lambda Labs is, “how do I track resource utilization for deep learning jobs?” Resource utilization tracking can help ...

TensorFlow 2.0 Tutorial 05: Distributed Training across Multiple Nodes

Distributed training allows scaling up deep learning task so bigger models can be learned or training can be conducted at a faster pace. In a previous ...

TensorFlow 2.0 Tutorial 04: Early Stopping

During training, weights in the neural networks are updated so that the model performs better on the training data. For a while, improvements on the training ...

TensorFlow 2.0 Tutorial 03: Saving Checkpoints

This tutorial combines two items from previous tutorials: saving models and callbacks. Checkpoints are saved model states that occur during training. With ...

TensorFlow 2.0 Tutorial 02: Transfer Learning

This tutorial shows you how to perform transfer learning using TensorFlow 2.0. We will cover:

A Gentle Introduction to Multi GPU and Multi Node Distributed Training

This presentation is a high-level overview of the different types of training regimes that you'll encounter as you move from single GPU to multi GPU to multi ...

Titan V Deep Learning Benchmarks with TensorFlow

In this post, Lambda Labs benchmarks the Titan V's Deep Learning / Machine Learning performance and compares it to other commonly used GPUs. We use the Titan V ...

RTX 2080 Ti Deep Learning Benchmarks with TensorFlow

Perform GPU, CPU, and I/O stress testing on Linux

CPU, GPU, and I/O utilization monitoring using tmux, htop, iotop, and nvidia-smi. This stress test is running on a Lambda GPU Cloud 4x GPU instance. Often ...

How to Run OpenAI's GPT-2 Text Generator on Your Computer

Update June 5th 2020: OpenAI has announced a successor to GPT-2 in a newly published paper. Checkout our GPT-3 model overview.

V100 server on-prem vs AWS p3 instance cost comparison

Deep Learning requires GPUs, which are very expensive to rent in the cloud. In this post, we compare the cost of buying vs. renting a cloud GPU server. We use ...

Install CUDA 10 on Ubuntu 18.04

You were probably thinking that this was going to be a long post. You're in luck. All you need to do is to install Ubuntu 18.04 and then Lambda Stack. Here's ...

Set up a GPU accelerated Docker container using Lambda Stack + Lambda Stack Dockerfiles on Ubuntu 20.04 LTS

Or, how Lambda Stack + Lambda Stack Dockerfiles = GPU accelerated deep learning containers

Text Generation: Char-RNN Data preparation and TensorFlow implementation

This tutorial is about making a character-based text generator using a simple two-layer LSTM. It will walk you through the data preparation and the network ...