Crowd Sourced Deep Learning GPU Benchmarks from the Community

October 12, 2018• 11 min read

We open sourced the benchmarking code we use at Lambda Labs so that anybody can reproduce the benchmarks that we publish or run their own. We encourage people to email us with their results and will continue to publish those results here. You can run the code and email benchmarks@lambdalabs.com or tweet @LambdaAPI. This is the official page for all Lambda Community Benchmarks.

How to get your results published here

• Download and run our benchmarking code: https://github.com/lambdal/lambda-tensorflow-benchmark
• Fill out this markdown table form for your system setup.

Component        | Version
-----------------|------------
CPU              | $(cat /proc/cpuinfo | grep 'model name' | uniq | awk -F: '{ print $2 }')
Distro          | $(lsb_release -d)
Kernel Version   | $(uname -r)
Kernel Arch      | $(uname -m)
GPU              | $(sudo lspci | grep VGA\ compat | head -n1)
Tensorflow       | $(python -c 'import tensorflow;print(tensorflow.__version__)' 2> /dev/null)
NVIDIA Driver    | $(head -n1 /proc/driver/nvidia/version | awk '{ print $8 }')
CUDA             | $(nvcc --version | tail -n 1 | grep Cuda | awk '{ print $6 }')
cuDNN           | $(cat /usr/include/cudnn.h | grep -P 'define\ CUDNN_MAJOR|define\ CUDNN_MINOR|define\ CUDNN_PATCHLEVEL' | awk '{ print $3 }' | sed ':a;N;$!ba;s/\n/./g')
Python           | $(python --version 2>&1)

Copy the above and paste into template.txt. Then run the code below to output your table.

IFS=' '
cat > template.txt
CTRL-V (paste in)
CTRL-D (end file)
(for line in $(cat template.txt); do eval "echo \"$line\""; done) > specs-table.txt

Mail the output summary.md file as well as the output system specification table above to benchmarks@lambdalabs.com.

Crowd Sourced Results

Here are the results that have been submitted to us by third parties.

Summary (Stanislav Brizitsky)

Summary

syn-replicated-fp32-1gpus

syn-parameter_server-fp32-1gpus

syn-replicated-fp16-1gpus

syn-parameter_server-fp16-1gpus

Summary (Antonio Marin)

RTX2080Ti benchmark

Specifications

Benchmark results

syn-replicated-fp32-1gpus

syn-parameter_server-fp32-1gpus

syn-replicated-fp16-1gpus

syn-parameter_server-fp16-1gpus

Summary - Mike Metral - 1080 Ti

syn-replicated-fp32-1gpus

syn-parameter_server-fp32-1gpus

syn-replicated-fp16-1gpus

syn-parameter_server-fp16-1gpus

Hardware / Software