Desktop windows 10 Cuda issue

post by Cherrypie on Jan 29, 2019

Hello. We recently bought the Desktop which has 4 titan xp.

I use the conda install tensorflow-gpu and keras.

However the kernel keeps dying.

The conda install the Cuda and cudnn install automatically.

However, it keeps dying so I try to install everything manually.

I have installed the Geforce and visual studio with 2015 toolkit.

However I face the cuda installation failure.

Also, the system bluescreen shows a lot.

Do you have any procedure manual for windows 10?

I need tensorflow and keras for it.

Thank you.

post by Cherrypie on Feb 6, 2019

I successfully installed cuda 9 and tensorflow GPU. However it freezes and shows the bluescreen whenever I used multi GPU training. I cannot believe that around 20,000 cad desktop has so many issues like this. My 2,000 asus laptop works without any problem. Do you guys have any support option instead of this board?

post by leonnowsden on Feb 6, 2019

Ehm this is looking like a BIOS failure. Use the default BIOS settings. Make sure that you installed the UEFI version of Windows and maybe reinstall all the drivers. Overall I can’t recommend a multi GPU rig in Windows for Tensorflow… it’s really buggy especially with conda. Install everything with pip and a UNIX based system. You will have a lot more luck with that.

Leon Nowsden

post by Cherrypie on Feb 6, 2019

Thank you Leon

I installed every driver already.

I face problems still.

First assumption is that we have 4 GPUs and I assigned manually 3 GPUs. When we use GPU 1 which seems like internally used in Windows (I checked by task manager and it goes 100% however others just use only 20% during the training) we face the freeze. So I assigned GPU ID only to use GPU 0, 2, 3. Now multi-GPU training is working.

This afternoon I will check again. In Windows, I can only use CUDA 9 for TensorFlow. I agree with your opinion however our organization cannot allow dual-booting. So I can only use the Windows computer in the office.

post by leonnowsden on Feb 9, 2019

Then I am sorry. I don’t have an idea right now.

Leon

post by sabalaba on Feb 9, 2019

@Cherrypie can you email support@lambdalabs.com

Off the top of my head it sounds like you’re using one GPU as the display GPU and then hammering on all four GPUs simultaneously, including the one that’s used to drive the display. Thus the freezing/stuttering.

Again, please submit this ticket to support@lambdalabs.com if you haven’t fixed it yet.