How to fix the GPU warning?

26 views (last 30 days)
Daigo
Daigo on 28 Dec 2021
Commented: Daigo on 2 Jan 2022
Hello,
I'm trying to use GPU libraries (especially, gpuArray) on an HPC cluster but I constatnly get an error below:
---
Warning: The CUDA driver must recompile the GPU libraries because your device is more recent than the libraries. Recompiling can take several minutes.
> In parallel.internal.gpu.selectDevice
In parallel.gpu.GPUDevice.current (line 44)
In gpuDevice (line 23)
---
Everytime I get this warning message, MATLAB stops working for more than 30 minutes, which is so annoying...
Information about the machine and MATLAB release is listed below:
  • MATLAB: R2019a
  • GPU chip: A100-PCIE-40GB
  • NVIDIA-SMI: 460.32.03 (Linux)
  • Driver version: 462.32.03
  • CUDA version: 11.2
I tried to disable the warning message by the following line of code:
warning off parallel:gpu:device:DeviceLibsNeedsRecompiling
but it didn't work.
Any idea how to solve this problem? I found some similar posts in MATLAB Community but I haven't found any helpful answer yet.
I'd appreciate any help.
  2 Comments
Matt J
Matt J on 28 Dec 2021
Have you tried updating or reinstalling your driver outside of Matlab?
Daigo
Daigo on 2 Jan 2022
Matt, thank you for your comment! Unfortunately, I cannot do that since the HPC cluster is managed by university. I will ask an system admin about this.

Sign in to comment.

Accepted Answer

Joss Knight
Joss Knight on 30 Dec 2021
It's all pretty clear from the documentation. Ampere cards are not supported in R2019a. You are using forward compatibility, which has limited capabilities and requires some tuning to ensure the lengthy compilation process doesn't happen more than once. On an HPC cluster, where you are not in control of the machine or which specific machine you will be allocated, you may need to ask your sysadmin to install the cuda cache binaries onto the cluster and set the relevant environment variables to avoid the recompilation.
Better still, get your cluster upgraded to the latest version of MATLAB to get full support for the GPUs it is using.

More Answers (0)

Tags

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!