Very Weird GPU Behavior

So I've been working with sending parts of my script's calculation to the GPU for analysis and I did not change the code a bit through this entire course of events.

I first made the switch over to using GPUs and suddenly my code was running 10 times faster and it was amazing. I felt like I was on top of the world but then suddenly I starting getting errors saying "CUDA_ERROR_OUT_OF_MEMORY" and my code would not run. I was able to get around this problem by littering my code with reset() commands but eventually my computer (in its entirety) crashed abruptly.

So I rebooted my computer and now parts of my code if I run them will return the error: "Error using gpuArray/size The data no longer exists on the device."

and even when I don't use this part of the code the rest of the code is now running 10 times slower than it was before I made the switch over to using the GPU.

I will be very honest; I have absolutely no idea what is happening right now. If I could reliably get the sort of speed increase I had initially though I would die a happy man.

Any help that anyone could offer and if anyone had an explanation of what was going on I would greatly appreciate it!

[Jacob Mevorach]

I was able to more or less solve the problem. I very infrequently get errors but the code runs very quickly. I needed to clear variables on the gpu after I used them to make it work.

i.e.:

 stuff = gpuArray(stuff);
Output = SomeFunctionThatIsCompatibleWithTheGPU(stuff);
newstuff = gather(Output);
clear stuff Output

Also I've been resetting the GPU using reset("GPU handle") at the start of my script.

Hope this is helpful to others!

[Joss Knight]

Resetting does a cudaDeviceReset and is rarely helpful for memory issues, especially as it sometimes doesn't explicitly delete GPU arrays from the workspace, so they become stale. As you correctly found, clearing variables using clear or clearvars is usually a good way to go for scripts. Ideally you'd be using functions that only create variables as they need them, because these tidy themselves up properly when they exit rather than littering the global workspace.

MATLAB does hold onto memory for some things like FFTs, and to make allocations quick and asynchronous. If you want to keep track of the free and available memory you can look at the FreeMemory and AvailableMemory properties of the gpuDevice:

   gpu = gpuDevice;
   freeMem = gpu.FreeMemory
   availMem = gpu.AvailableMemory

Free memory is space left on the GPU. Available memory is that plus anything that MATLAB is holding onto, but could possibly free up if needed.

[Jacob Mevorach]

Thank you. I appreciate the knowledge.