Why do I need 'wait' commands to assess GPU performance in the Parallel Computing Toolbox 6.0 (R2012a)?

The following script was used to compare performance of GPU vs CPU:

dev = gpuDevice;

wait(dev);

Narr=[1024:1024:5*1024];

for i = 1:numel(Narr)

      N = Narr(i);

      A=rand(N,N);

      tic;

      x1=fft(A);

      cpuTime=toc;

      Ag=gpuArray(A);

      wait(dev);

      tic;

      x2=fft(Ag);

      wait(dev);

      gpuTime1=toc;

      fprintf('Size = %d, speedup = %f\n',N,cpuTime/gpuTime1)

end

Why do we need to use the 'wait' function when timing GPU performance while this was not the case in MATLAB 7.13 (R2011b) and prior releases?

Answer 1

[MathWorks Support Team]

In MATLAB 7.14 (R2012a), GPU operations are asynchronous from CPU operations which means that MATLAB continues while the GPU is running. Thus, when assessing GPU performance, we need to include 'wait' commands to ensure that the GPU completed its work. Without waiting on the GPU, the tic/toc results do not make sense.

 In previous releases (R2011b and earlier), MATLAB and the GPU were synchronous, so that any calls to the GPU had to complete before MATLAB proceeded to the next command.

This information is in the R2012a release notes as well:

http://www.mathworks.com/help/releases/R2012a/toolbox/distcomp/rn/bs8h9g9-1.html

[Noam]

I have matlab 2013a. The code is asynchronous but the 'wait' command came only in 2013b.

How can I tic-toc an opertaion on the gpu?

For now i'm doing this trick:

tic;

ZZZ=ZZ*Z; % operation that I want to check its running time.

ZZZ(1) = ZZZ(1); % "wait"

toc

[Jill Reese]

Noam, as mentioned in the above answer, the wait command for gpuDevice was introduced in MATLAB R2012a.

Please click on the link for the R2012a release notes; the section on GPU computing discusses the changes related to asynchronous execution and the wait command. You can also find help directly for the wait command in the release you are using (R2013a) by typing

help parallel.gpu.GPUDevice/wait

[Noam]

Thanks. I'm pretty sure that it didn't work yesterday (~_~;)

[Jan]

For users of this forum, who are not associated to a specific license, the posted link leads to the "Access to archived documentation is a benefit for licensed users of MathWorks products" page. What a pitty.

Answer 2

[Ben Tordoff]

If you have R2013b or above, an even better way to measure the time taken for a function is to use GPUTIMEIT (for GPU code that doesn't "gather" at the end) or TIMEIT (for all other MATLAB code). These helpers will run the function you give them multiple times and do all the right waiting etc. to get accurate timing results.