Better performace: GPU + Workers

How can I know what is the best optimization method for me?

1. Run the code only inside the GPU? Or run only inside the multiple workers/cores?
2. Is possible to join/mix theses two methods to get a even better performance?

Answer 1

[Joss Knight]

It depends what you're optimizing. Use of the GPU only really make sense if the objective function is a large enough operation to fully utilize the GPU (e.g. it is multiplying a very large matrix by a vector, such as in iterative solvers). Use of a parallel pool with GPU computation only gains you anything if you have multiple GPUs, but is perfectly possible if you are implementing your own objective function.

Alternatively, if you are on linux, you can try using MPS to allow overlapping use of the same GPU on multiple workers. This can potentially make it viable to use the GPU with smaller operations.

[Nycholas Maia]

Why you said "if the objective function is a large enough operation to fully utilize the GPU".

Is not good always use GPU in small or medium operations too?

[Walter Roberson]

"Is not good always use GPU in small or medium operations too?"

Not always. There is communications overhead with the GPU.

[Joss Knight]

Well, my answer to that would be no, the GPU SMs are a lot slower than a CPU core, so you need a lot of parallelism to make it worthwhile.