Nvidia-smi freezes/hangs indefinitely

We purchased a Lambdal Quad earlier this summer. It has been working just fine for months, but all of a sudden, yesterday it broke: when we type nvidia-smi, it hangs/freezes indefinitely! We also can’t import tensorflow or torch (both of them also hangs indefinitely).

Update

After waiting for ...

Failed to initialize NVML: Driver/library version mismatch

Hi Stephen, the simple solution worked! Thank you!

Failed to initialize NVML: Driver/library version mismatch

Hi, I’m having this problem in our machine: Failed to initialize NVML: Driver/library version mismatch

Here’s our folder structure:

anie@arthur1:~$ ls /usr/local | grep cuda
cuda
cuda-8.0

Here’s the lib64 for cuda folder:

anie@arthur1:~$ ls /usr/local/cuda/lib64
libcublas_device.a libcufft.s...