# Hyperplane Setup

## post by jeremy on Oct 5, 2020

## Hyperplane Setup

### Mellanox OFED, NVIDIA Peer Memory, NVIDIA Driver, CUDA, cuDNN & Docker

Download the [Mellanox OFED tarball](https://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers&mrequest=downloads&mtype=ofed&mver=MLNX_OFED-5.1-2.3.7.1&mname=MLNX_OFED_LINUX-5.1-2.3.7.1-ubuntu18.04-x86_64.tgz)
and copy it to `/tmp`. We’ll need it later:

```sh

cp MLNX_OFED_LINUX-5.1-2.3.7.1-ubuntu18.04-x86_64.tgz /tmp
```

#### NVIDIA Peer Memory

Clone the NVIDIA Peer Memory repository & compile it:

```sh

git clone https://github.com/Mellanox/nv_peer_memory -b 1.0-9
cd nv_peer_memory
./build_module.sh
mv /tmp/nvidia-peer-memory_1.0.orig.tar.gz .
tar xzf nvidia-peer-memory_1.0.orig.tar.gz
cd nvidia-peer-memory
dpkg-buildpackage -us -uc
```

And copy the resulting debian files to `/tmp` also:

```sh

sudo cp ../*.deb /tmp
```

Then download the `hyperplane_install.sh` script to, saving it to `/tmp`. Then run it:

```sh

cd /tmp
wget files.lambdalabs.com/scripts/hyperplane_install.sh
chmod +x hyperplane_install.sh
sudo ./hyperplane_install.sh
```

* * *

### Fabric Manager

The Fabric Manager is available in NVIDIA’s machine learning repo (if you’ve run `hyperplane_install.sh` above, the repo is already added).

Once you add the repos & trust the keys, you can install the Fabric Manager like so:

```sh

sudo apt -y install nvidia-fabricmanager-450
```

* * *

### DCGM

You can download the Data Center GPU Manager debian file [here](https://developer.nvidia.com/dcgm#Downloads) (requires NVIDIA account), then it can be installed with:

```sh

sudo dpkg -i datacenter-gpu-manager_2.0.13_amd64.deb
```

* * *

### Apt Mirrors

`apt-mirror.sh` configures `nginx` and `apt-mirror`.

```sh

wget files.lambdalabs.com/scripts/apt-mirror.sh
sh apt-mirror.sh
```

`apt-sources.sh` installs repos for the client machine. Pass it the hostname or IP of your `apt-mirror` server:

```sh

wget files.lambdalabs.com/scripts/apt-mirror.sh
sh apt-mirror.sh <apt-mirror host>
```

* * *

### Apt

Though adding these is unnecessary if you’ve run `hyperplane_install.sh`, we’re including these for clarity.

Repositories:

```sh

deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu18.04/x86_64 /
deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu18.04/x86_64 /
deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
deb https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-container-runtime/stable/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-docker/ubuntu18.04/$(ARCH) /
```

Public keys for the repos:

```sh

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
sudo apt-key adv --fetch-keys "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub"
```