How do I use Multi-Instance GPU (MIG) mode GPUs with MATLAB for computation?

Nvidia announced a new feature which allows larger server GPUs like the A100 to be securely partitioned into multiple "GPU instances" and "Compute Instances" for CUDA applications, providing multiple users with separate GPU resources. I would like to use a card divided in this way with MATLAB, how can I achieve this?

Answer 1

[MathWorks Support Team]

As of MATLAB R2021a, there is some limited support added to MATLAB for using MIG mode GPUs. MIG mode GPUs are currently only supported by Nvidia on Linux OS.

 

Please be aware that the "gpuDeviceCount" function will not detect the GPU or Compute Instances as separate devices. As a result, the "gpuDeviceCount" function returns as output the number of physical GPUs without counting the MIG partitions.

 

If you wish to use a parallel pool with each worker using a different GPU or Compute Instance on the MIG mode GPU then you will need to override the default GPU device selection the "parpool" initializes with. This can be achieved by setting the "CUDA_VISIBLE_DEVICES" environment variable on each worker before any GPU operations are executed:

1. Execute "nvidia-smi -L" in a Linux terminal on the machine to get the complete name of the GPU.\n
   • This should be in the form of: MIG-<UUID>
   •  e.g. MIG-d9a47cfb-d1f2-d08c-b40e-0812b597ae69
2. Before running anything else on the MATLAB workers, you need to set the environment variable "CUDA_VISIBLE_DEVICES" with the names acquired from step 1 in an "spmd" block to assign the GPU or Compute Instances to each process.

parpool("local",2) 
spmd
    if spmdIndex == 1 % assign one GPU instance to worker 1
        setenv('CUDA_VISIBLE_DEVICES', "MIG-GPU-d9a47cfb-d1f2-d08c-b40e-0812b597ae69");
    else if spmdIndex == 2 % assign a different GPU instance to worker 2
        setenv('CUDA_VISIBLE_DEVICES', "MIG-GPU-g4j34dbz-g6t7-h42f-k97d-87f5623s34fd");
    end
end