Beeping sound during model inference

post by wh123 on Sep 5, 2024

I currently have a Vector Pro with 2xA6000s. When I send a request to the GPUs to do inference, I get a beeping sound that lasts for 1-2 seconds. This only happens for large model inference/more intensive computation.

The weights are pre-loaded into the devices so the beeping sound only originates when actual inference occurs. There is also no spike in temperatures. According to nvidia-smi, the GPUs are around 65C when the beeping sound occurs.

Any help is greatly appreciated. Thanks!

80 views