Configure GPU Support in MATLAB Online Server

Adding GPU support to your MATLAB^® Online Server™ cluster can speed up MATLAB code for your end users.

Enable GPU

To enable GPU support, create a GPU node in your Kubernetes^® cluster and configure your MATLAB resource pools to deploy to that node.

Verify that the machine running MATLAB Online Server has a supported NVIDIA^® graphics card and drivers installed. To confirm that your card and drivers are compatible with the MATLAB versions that require GPU support, see GPU Computing Requirements (Parallel Computing Toolbox).
Enable GPU support in your MATLAB Online Server cluster by following the instructions provided in the NVIDIA GitHub^® repository for the Kubernetes device plugin.
1. Verify the prerequisites listed in Prerequisites.
2. Prepare your GPU node by following the Preparing Your GPU Nodes instructions.
3. Enable resources to use this GPU node by following the Enabling GPU Support in Kubernetes instructions.
Verify that the previous step enabled GPU support. On your MATLAB Online Server machine, run these commands.
```
kubectl get daemonset --namespace kube-system
kubectl get pods --namespace kube-system
```
Confirm that the returned daemonset and pod output includes the text "nvidia-device-plugin".
Configure the MATLAB resource pool to run on the enabled GPU node. In the <server-root>/overrides/<cluster-name>/<namespace-name>/matlab-pool.yaml file of your MATLAB Online Server root folder, add these YAML fields. If your cluster contains multiple MATLAB pools, then in the <server-root>/overrides/<cluster-name>/<namespace-name>/matlab-pool folder, set these fields instead in the YAML file of the MATLAB pool that requires GPU support.
replicaCount: 1 resourceQuota: matlab-pool-matlab: limits: nvidia.com/gpu: 1
If your cluster runs more than one GPU, then increase the value of the nvidia.com/gpu field to match the number of running GPUs.
Apply your changes by undeploying and redeploying the MATLAB pool service.
```
./mosadm undeploy matlab-pool
./mosadm deploy matlab-pool
```
Confirm that GPU support is enabled in MATLAB Online™. Open MATLAB Online from a browser at the URL for your organization. For example: https://matlab.your-organization.com. Then, run this MATLAB command:
```
gpuDevice
```
Verify that your GPU device is listed. If you do need see your device listed or you see any other unexpected output, contact technical support. See Contact Technical Support About MATLAB Online Server Issues.

Your cloud vendor might have additional requirements for enabling GPUs. For more details, see the documentation for your cloud vendor. For example:

Amazon^® EKS — https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-eks-setup.html
Azure^® AKS — https://learn.microsoft.com/en-us/azure/aks/gpu-cluster
Google^® GCP — https://cloud.google.com/kubernetes-engine/docs/how-to/gpus

Manage GPU Access

To restrict access to the GPU only to the MATLAB pool pods running on your machine, you can use Kubernetes taints and tolerations.

A taint applies to a node and restricts pods from accessing that node. Set a taint that restricts pods from scheduling onto your GPU node by specifying the NoSchedule effect.

kubectl taint nodes <my-gpu-node> <gpu-key>=<gpu-value>:NoSchedule

<my-gpu-node> is the name of the GPU node running on the Kubernetes cluster.

<gpu-key>=<gpu-value> specifies that only pods with the specified matching key-value pair, called a toleration, can access this node.

Specify a toleration to enable a MATLAB pool pod to access the tainted GPU node. In the <server-root>/overrides/<cluster-name>/<namespace-name>/matlab-pool.yaml file of your MATLAB Online Server root folder, add a tolerations section. Specify the matching key-value pair and NoSchedule effect that you applied to the GPU node. If your cluster contains multiple MATLAB pools, then in the <server-root>/overrides/<cluster-name>/<namespace-name>/matlab-pool folder, set these fields instead in the YAML file of the MATLAB pool that requires GPU support.

tolerations:
  - key: "<gpu-key>" 
    operator: "Equal" 
    value: "<gpu-value>"
    effect: "NoSchedule"

To apply your changes, undeploy and redeploy the MATLAB pool service.

./mosadm undeploy matlab-pool
./mosadm deploy matlab-pool

For more details on setting taints and tolerations, see Taints and Tolerations in the Kubernetes documentation.