exp function on single gpuArray vs exp function on single normal array precision difference

Hello,

I am using actually gpuArray to optimize my computation with a Graphic Card : either NVidia GeForce GTX680 or NVidia Tesla K40M.

I am facing a strange behaviour related to precision with the exp function, which introduce some precision problem in a gpu context :

code :

tested = single([0.0001    0.0218    0.0062    0.0011   -0.0023    0.0097    0.0001    0.0000   -0.0022   -0.0011]);
tested_gpu = gpuArray(tested);
exp_tested_gpu = exp(tested_gpu);
exp_tested = exp(tested);
exp_tested_gpu_gathered = gather(exp_tested_gpu);
exp_tested - exp_tested_gpu_gathered

=> result : 
ans =

    1×10 single row vector

     1.0e-06 *

           0         0         0         0    0.0596   -0.1192         0         0         0         0

When I use double everything seems to be ok :

code :

tested = double([0.0001    0.0218    0.0062    0.0011   -0.0023    0.0097    0.0001    0.0000   -0.0022   -0.0011]);
tested_gpu = gpuArray(tested);
exp_tested_gpu = exp(tested_gpu);
exp_tested = exp(tested);
exp_tested_gpu_gathered = gather(exp_tested_gpu);
exp_tested - exp_tested_gpu_gathered

result => 

ans =

       0     0     0     0     0     0     0     0     0     0

Is it a way to handle this precision problem?

Answer 1

[Joss Knight]

Your test does not do a fair comparison, since both the host and GPU versions should be compared against the double-precision version. If you do this you'll see that both have inaccuracies, but they are at the level of the accuracy of single precision numbers themselves.

MATLAB Math uses a specialised version of single precision exp that is optimised for maximum accuracy. gpuArray uses the version of exp that comes from the NVIDIA libraries, which is optimised for performance. The NVIDIA version is actually more accurate than the version that comes with the C standard math libraries, but less accurate than core MATLAB. I haven't looked into whether we could implement the same algorithm as core MATLAB without making exp slower, but it certainly seems likely it would. Ultimately it depends on whether accuracy or performance is more important. For best accuracy people might be expected to use double precision, so for single it's probably fair to say performance is more important, but every application will have different priorities.

This plot shows the distribution of error in the result for values input to exp between -0.1 and 0.1, using core MATLAB, gpuArray, and the C standard library.