Using SkyPilot to deploy a Kubernetes cluster - Lambda Docs

Using SkyPilot to deploy a Kubernetes cluster

Introduction

SkyPilot makes it easy to deploy a Kubernetes cluster using Lambda Cloud on-demand instances. The NVIDIA GPU Operator is preinstalled so you can immediately use your instances' GPUs.

In this tutorial, you'll:

You're billed for all of the time the instances are running.

All of the instructions in this tutorial should be followed on your computer.

This tutorial assumes you already have installed:

You can install these packages by running:

sudo apt update && sudo apt install -y python3 python3-venv python3-pip curl netcat socat

You also need to install kubectl by running:

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && \
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Configure your Lambda Cloud firewall and generate a Cloud API key

In the Firewall settings page of the Lambda Cloud console, add rules allowing incoming traffic to ports TCP/443 and TCP/6443.

Generate a Cloud API key for SkyPilot. You can also use an existing Cloud API key.

Install SkyPilot

Create a directory for this tutorial and change into the directory by running:

mkdir ~/skypilot-tutorial && cd ~/skypilot-tutorial

Create and activate a Python virtual environment for this tutorial by running:

python3 -m venv ~/skypilot-tutorial/.venv && source ~/skypilot-tutorial/.venv/bin/activate

Then, install SkyPilot in your virtual environment by running:

pip3 install skypilot-nightly[lambda,kubernetes]

Configure SkyPilot for Lambda Cloud

Download the SkyPilot example cloud_k8s.yaml and launch_k8s.sh files by running:

curl -LO https://raw.githubusercontent.com/skypilot-org/skypilot/master/examples/k8s_cloud_deploy/cloud_k8s.yaml && \
curl -LO https://raw.githubusercontent.com/skypilot-org/skypilot/master/examples/k8s_cloud_deploy/launch_k8s.sh

Edit the cloud_k8s.yaml file.

At the top of the file:

It's important that you use a strong passphrase. Otherwise, the Kubernetes cluster can be compromised, especially if your firewall rules allow incoming traffic from all sources.

You can generate a strong passphrase by running:

openssl rand -base64 16

This command will generate a random string of characters such as zPUlZGe4HRcy+Om04RvGmQ==.

The top of the cloud_k8s.yaml file should look similar to:

resources:
  cloud: lambda
  accelerators: H100:1

num_nodes: 2

envs:
  SKY_K3S_TOKEN: zPUlZGe4HRcy+Om04RvGmQ== # Can be any string, used to join worker nodes to the cluster

You can set accelerators to a different instance type, for example, A100:8 for an 8x A100 instance or H100:8 for an 8x H100 instance.

Create a directory in your home directory named .lambda_cloud and change into that directory by running:

mkdir -m 700 ~/.lambda_cloud && cd ~/.lambda_cloud

Create a file named lambda_keys that contains:

api_key = API-KEY

Replace API-KEY with your actual Cloud API key.

Use SkyPilot to launch instances and deploy Kubernetes

Change into the directory you created for this tutorial by running:

cd ~/skypilot-tutorial

Then, launch 2 1x H100 instances and deploy a 2-node Kubernetes cluster using those instances by running:

bash launch_k8s.sh

You'll begin to see output similar to:

===== SkyPilot Kubernetes cluster deployment script =====
This script will deploy a Kubernetes cluster on the cloud and GPUs specified in cloud_k8s.yaml.

+ CLUSTER_NAME=k8s
+ sky launch -y -c k8s cloud_k8s.yaml
Task from YAML spec: cloud_k8s.yaml
...

It usually takes about 15 minutes for the Kubernetes cluster to be deployed.

The Kubernetes cluster is successfully deployed once you see:

🎉 Enabled clouds 🎉
  ✔ Kubernetes
  ✔ Lambda
+ set +x
===== Kubernetes cluster deployment complete =====
You can now access your k8s cluster with kubectl and skypilot.

To test the Kubernetes cluster, launch a job by running:

sky jobs launch --gpus H100 --cloud kubernetes -- 'nvidia-smi'

You'll see output similar to the following and will be asked if you want to proceed:

Task from command: nvidia-smi
Managed job 'sky-cmd' will be launched on (estimated):
...

Press Enter to proceed.

You should see output similar to the following, indicating the job ran successfully:

Launching managed job 'sky-cmd' from jobs controller...
...

Useful Commands

Managed Job ID: 1