Is Matlab/ Simulink supporting Nvidia external GPU Systems/Thunderbold 3 for Machine Learning and Deep Learning projects including embedded systems like Jetson TX2?

Hi there,

we are planning some research projects on machine learning (big data approaches, creating predictive models, deep learning with object recognition). We plan to use GPU computing for this.

To make the GPU easily available for different users (with their laptops) we thought of buying an "external GPU" case (equiped with i.e. Nvidia Geforce 1080Ti) in combination with Thunderbold 3 interfaces. Different users could prepare their tasks and alternately connect the eGPU only for training of their networks.

Does MATLAB/Simulink support this setup?

Does anybody use this hardware setup? (-in combination with the "GPU-Coder Add-on" and Nvidia Jetson TX2 embedded hardware?)

How are your experiences?

Thank you!

Marcus

Answer 1

[Joss Knight]

Yes, it's supported. This isn't really a question of MATLAB's system requirements; if the CUDA driver can interact with your card then so can MATLAB. You will notice a considerable increase in latency due to the dramatically reduced bandwidth to get data on and off the card, so you need to minimize the amount of CPU/GPU communication.

Answer 2

[Marcus Kreuzer]

Thank you for this information!

Since I'm a newbie to this topic: I asked myself what you mean with "considerable" increase in latency? Could you please give a rough estimation? Let's say, a comparison (same GPU for all) of:

(i) a desktop system with internal GPU (calculation needs 100min)

(ii) an eGPU system with optimized CPU/GPU communication (estimated calculation needs 100+x min)

(iii) an eGPU system with non-optimized CPU/GPU communication (estimated calculation needs 100+evenmorethan x min)

For example, a Nvidia GTX 1080 Ti GPU card is equipped with 11GB of V-RAM. So to reduce the CPU/GPU communication the aim is to fill this V-RAM with data, let the GPU do the calculations, and "bring back" the results to the CPU/PC, right? Or do CPU and GPU have to communicate continously and establish a kind of a data stream?

Thank you!

Marcus

[Joss Knight]

Hi Marcus. All CPU/GPU communication is slowed down by an order of magnitude, so that means data transfer but also simply invoking a kernel launch for instance. So if you need to wait for the result of a computation, do something on the CPU, then launch another kernel, you incur the full round-trip cost. Hybrid algorithms that use CPU and GPU like SVD and other matrix factorizations suffer from this in particular.

In a particular bit of benchmarking I did (which you should take with a pinch of salt because it was one particular setup) I found that CPU/GPU bandwidth was 8x slower. So MLDIVIDE was 5x slower; but MTIMES and FFT were unchanged. Deep Learning training of AlexNet was about 2x slower. Worth noting that everything was of course still a lot faster than just running on the CPU.

The faster the connection gets (USB C Thunderbolt) the less of a problem this is so maybe one day it won't be so noticeable, if you stick to this solution.

[Marcus Kreuzer]

Hi Joss, thank you very much again for your answer! I didn't expect the impact to be so big. For me, it now looks like a solution with internal GPU would be more effective at the end, even if it's a bigger investment (new desktop PC compared to single eGPU case) at the beginning.

May you can answer one last question: Which interface (Thunderbolt 3 ?) / hardware case did you use for your tests? Did the case and your PC work with PCIe 4x (or 2x, 1x)?

Thank you!

Marcus

[Joss Knight]

I have experimented with the Razer Core with a USB C connection to a Lenovo laptop (can't remember which one) - however, I'd urge you not to make purchasing decisions on the basis of my one data point taken a year ago!

[Walter Roberson]

I was looking at some Thunderbolt 3 external cards yesterday in response to someone else interested in external GPU enclosures. It turned out that there were two basic kinds:

1. ASUS custom connectors, only compatible with some ASUS models; Or
2. PCI express x4 or higher.

[Marcus Kreuzer]

Hi!

I wanted to give some feedback:

We finally decided to buy a workstation, equipped with a Nvidia GTX 1080 Ti. The reason is simple: an external case is pricy, for a little more you get an Desktop PC, too. And laptops with Thunderbolt C interface are still rare. As soon as we bring this card to its limits (no one knows when...), we'll buy the "latest" GPU by then. The old one will be put into an external case to be used for "first experiments" and student's practices.

Another interesting source may be: https://egpu.io/build-guides/

This community did a lot of work for using external GPUs on Laptops (Tutorials, lists with working "constructions" etc...)

Thank you all for your help!

Marcus

[Walter Roberson]

Thanks for the update, Marcus.