Failed to initialize NVML: Driver/library version mismatch

post by anie on Jan 12, 2018

Hi,

I’m having this problem in our machine:

Failed to initialize NVML: Driver/library version mismatch

Here’s our folder structure:

anie@arthur1:~$ ls /usr/local | grep cuda
cuda
cuda-8.0

Here’s the lib64 for cuda folder:

anie@arthur1:~$ ls /usr/local/cuda/lib64
libcublas_device.a   libcufft.so           libcusolver.so.8.0.61  libnppicom.so.8.0.61  libnppist.so.8.0      libnvgraph_static.a
libcublas.so         libcufft.so.8.0       libcusolver_static.a   libnppidei.so         libnppist.so.8.0.61   libnvrtc-builtins.so
libcublas.so.8.0     libcufft.so.8.0.61    libcusparse.so         libnppidei.so.8.0     libnppisu.so          libnvrtc-builtins.so.8.0
libcublas.so.8.0.61  libcufft_static.a     libcusparse.so.8.0     libnppidei.so.8.0.61  libnppisu.so.8.0      libnvrtc-builtins.so.8.0.61
libcublas_static.a   libcufftw.so          libcusparse.so.8.0.61  libnppif.so           libnppisu.so.8.0.61   libnvrtc.so
libcudadevrt.a       libcufftw.so.8.0      libcusparse_static.a   libnppif.so.8.0       libnppitc.so          libnvrtc.so.8.0
libcudart.so         libcufftw.so.8.0.61   libnppc.so             libnppif.so.8.0.61    libnppitc.so.8.0      libnvrtc.so.8.0.61
libcudart.so.8.0     libcufftw_static.a    libnppc.so.8.0         libnppig.so           libnppitc.so.8.0.61   libnvToolsExt.so
libcudart.so.8.0.61  libcuinj64.so         libnppc.so.8.0.61      libnppig.so           libnpps.so            libnvToolsExt.so.1
libcudart_static.a   libcuinj64.so.8.0     libnppc_static.a       libnppig.so.8.0.61    libnpps.so.8.0        libnvToolsExt.so.1.0.0
libcudnn.so          libcuinj64.so.8.0.61  libnppial.so           libnppim.so           libnpps.so.8.0.61     libOpenCL.so
libcudnn.so.5        libculibos.a          libnppial.so.8.0       libnppim.so.8.0       libnpps_static.a      libOpenCL.so.1
libcudnn.so.5.1.10   libcurand.so          libnppial.so.8.0.61    libnppim.so.8.0.61    libnvblas.so          libOpenCL.so.1.0
libcudnn.so.6        libcurand.so.8.0      libnppicc.so           libnppi.so            libnvblas.so.8.0      libOpenCL.so.1.0.0
libcudnn.so.6.0.21   libcurand.so.8.0.61   libnppicc.so.8.0       libnppi.so.8.0        libnvblas.so.8.0.61   stubs
libcudnn.so.7        libcurand_static.a    libnppicc.so.8.0.61    libnppi.so.8.0.61     libnvgraph.so
libcudnn.so.7.0.5    libcusolver.so        libnppicom.so          libnppi_static.a      libnvgraph.so.8.0
libcudnn_static.a    libcusolver.so.8.0    libnppicom.so.8.0      libnppist.so          libnvgraph.so.8.0.61

I’m wondering if you guys have encountered this problem before, and what’s the best way to fix it…(there are many solutions all over the internet…)

post by sabalaba on Jan 12, 2018

OK, first step, can you perform a hard reboot on your machine?

sudo reboot

If that doesn’t work, see here: CUDA NVIDIA NVML Driver/Library version mismatch - update-initramfs -u not working - old kernel version - Technical Help - DeepTalk - Deep Learning Community

post by anie on Jan 12, 2018

Hi Stephen, the simple solution worked! Thank you!