Double vs Single Precision, Matlab in 2019

Matlab defaults to double precision, but single precision is sufficient for many computational problems. In addition, single precision uses half the memory, and is generally twice as fast.

To convert from double to single precision is easy, in that all that is needed is to define the starting variables as single, and then all subsequent variables will default to the variable in each calculation with the lowest precision.

The best approach to define type single is:

X = zeros(10,10, 'single');

The construction: X = single(zeros(10,10)); is four times slower!!

Note that there is no difference between these two constructions for type double given that double precision is the default.

PROBLEMS

I ran into two problems that dimmed my enthusiasm for single precision.

1) Some Matlab functions will only work with double precision. The interpolation functions are the main examples that I have come across. (It would be great to have some documentation, either informal or formal, listing those Matlab functions that have type limitations. I have yet to find this information on the web.) To use the interpolation functions, you have to convert the input arguments to double, and then convert the output arguments back to single, which is cumbersome, and perhaps troublesome as well, as noted below.

2) A second problem is that the transition back and forth between single and double can cause errors in the limits for the interpolation. I ran across an error where the interpolation started to return nans after conversion to single precision. This problem would not have occured if the interpolation functions were able to work with single precision. That said, it might be possible to set the extrapolation option to "nearest" in order to navigate around the small round-off errors associated with the transition between single and double precision. Of course, this option could only be trusted when the algorithm was known to be fully functional.

For now, I have decided to stick with double precision.

I found very little discussion of this issue on the web. Surely there are others out there with experiences and recommendations about single precision.

Best,

Mark

Answer 1

[John D'Errico]

This topic is worth discussing.

X = zeros(10,10, 'single');
The construction: X = single(zeros(10,10)); is four times slower!!

Of course it is! Why would you not expect that? The former just fills the assigned memory locations with the same zero element. The latter fills it with double precision zeros, then needs to move the entire array to a new location, converting each element to a single on the fly.

"In addition, single precision uses half the memory, and is generally twice as fast."

Had you stated that it is SOMETIMES twice as fast, you would be correct. Newer releases have improved in this respect, although users wth older releases may find that single is no faster than double computations.

The problem is that single precision can be dangerous. You may not always know when you are near the edge of a numerical cliff. Using doubles keeps you much farther away from that edge. Yes, good numerical analysis, good numerical methods is a good thing. But if you use singles just to save some CPU time and some memory, then you are forgetting a major thing - CPU time is cheap, as is memory. So if you are using singles just to be frugal, then you are making a mistake. Use single precision when you absolutely need to do so, and only when you have the numerical skills to know that you can safely afford the lower precision.

Effectively, if you use single precision for no valid reason than pure frugality, then you are pre-optimizing your code, often a bad thing.

"Matlab defaults to double precision, but single precision is sufficient for many..."

A better way to say that is:

Matlab defaults to double precision, but you can sometimes survive the use of single precision. As the precision gets smaller, the risks grow greater.

[Mark Brandon]

John,

Thanks for your comments. Yes, memory and speed are cheap now, but that is often matched by an interest in larger problems.

My motivation is to keep a very large parallel computation within the time and memory bounds of the computer cluster I am using. The scale of the problem has caused me to move from 24 hours with 20 workers, to 48 hours with 20 to 28 workers. So, I was looking for some way to get back to a faster and smaller computation.

Over the last 15 years, I have experimented with single precision twice. Yes, you are right: in the dark old days, there was no difference in speed between single and double precision. But my experiments, and others, indicates about a factor two difference in speed with the modern version of Matlab. So that extra speed might be useful in some cases. The computation I am doing does not require double precision, so I decided see if single precision would work. Compared to the past, almost all Matlab functions support multiple types, except for the interpolation functions. My problem was mostly with the scatteredIntepolation algorithm.

So, I am back to using double precision....

Best,

Mark

[Walter Roberson]

Several Intel chips are compared in http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ .

The pattern is fairly consistent: single precision addition is notably faster than double precision addition, but single precision multiplication is slower than double precision multiplication, or at most barely faster (older chips.)

Note, though, that this is for CPUs. The relative performances of single and double precision on NVIDIA GPU is considerably more complicated. Single precision on GPU can be 3 times, 8 times, 24 times, or 32 times faster than double precision, depending on the NVIDIA GPU model.

[Dan Bindman]

Thanks Mark, John, and Walter, this thread was very useful to me.

I am in a similar situation as Mark, I do custom iterative fitting on very large data sets that can push close to the max in memory (currently at 64GB) and the fact that "single" looks like it can double my speed and can half my memory requirements is absolutely huge! And for the type of things I am running, the risks John points out aren't really in play.

But now just seeing Mark's update, where he said going to "single" didn't make much of a difference. I am still going to try it though. Based on some preliminary experimentation, I am getting good speed improvements with "single", so maybe it depends on the types of computations being done, as has already been mentioned.

[Steven Lord]

If you're working with data that's pushing (or pushing past) the limits of the amount of memory you have on your machine, you may want to explore some of the tools for working with Big Data that are included in MATLAB. Depending on what functions and functionality you're using for your iterative fitting making your data tall might just plain work with no (or little) modification to your existing code.

Alternately if your data has a lot of 0 elements, consider sparse matrices.

Answer 2

[Krishna Bindumadhavan]

Although this topic is about single precision, another floating point type with reduced precision that we are actively working on supporting with MATLAB is the half precision data type available with the fixed point designer toolbox:https://www.mathworks.com/help/fixedpoint/ref/half.html. With GPU Coder, you can deploy trained neural networks with half precision optimizations from 19a onwards.

Although half is usually not suitable for general purpose scientific computing, several applications like deep learning (training + inference) and image processing have proven benefits for using half precision to reduce memory bandwidth and computation time if the application can tolerate the reduced precision (as usually the case with deep learning).

On recent GPU's from NVIDIA like the Turing and Volta series, there is dedicated support for half precision in the hardware via tensor cores, which can accelerate computations like matrix multiply up to 8x.

This area is under active development and we expect to improve and expand support for half in various products in coming releases.

Answer 3

[Walter Roberson]

>> x = linspace(single(0),single(pi),20);

>> y = single(rand(1,20));

>> z = interp1(x, y, linspace(single(0),single(1),20))

z =

1×20 single row vector

Columns 1 through 9

0.7323988 0.5099056 0.2874124 0.06491931 0.2007067 0.3955918 0.5904768 0.6504743 0.6571828

Columns 10 through 18

0.6638914 0.6852249 0.7173581 0.7494914 0.7815178 0.8134047 0.8452916 0.8761108 0.9043419

Columns 19 through 20

0.9325731 0.9475382

That looks like it supports single to me.

https://www.mathworks.com/help/matlab/ref/interp1.html#btwp6lt-1-x

x— Sample points

vector

Data Types: single| double| duration| datetime

That looks like it supports single to me.

Answer 4

[Mark Brandon]

Walter,

Yes, you are right... interp1 is supported for a range of types. I should have been more precise, stating only that some of the interpolation schemes are not supported. My experience is with scatteredInterpolant, which can be used by itself and is also invoked internally in some other interpolation functions.

I had thought, mistakenly, that Matlab was now fully converted to single and double types, but that hunch was wrong. In addition, I started to realize that I would have to look up every function that I am using to see if the single type is supported. There is no list of functions that are doubles only.

I should note that Matlab is my favorite computational tool, and I have been using it in my research for 15 years now. I found this particular issue of the "singles game" to be rather poorly documented. Many people have suggested that the single precision is a useful method for getting a smaller and faster computation. I agree with John D'Errico that there are dangers involved in using single precision. Nonetheless, I am looking to get suggestions from people who have used this approach for large computations. What are the problems and does the outcome justify the work involved in the transition?

Best,

Mark

p.s. Thanks to you and John for your comments. The two of you are among my favorites at MATLAB Answers!

https://www.mathworks.com/help/matlab/ref/scatteredinterpolant.html

Input Arguments

expand all

x, y, z — Sample points

vector

Sample points, specified as vectors of the same size as v. The sample points should be unique. However, if the sample points contain duplicates, scatteredInterpolant displays a warning and merges the duplicates into a single point.

Data Types: double

P — Sample points array

matrix

Sample points array, specified as an m-by-n matrix, where m is the number of points and n is the dimension of the space where the points reside. Each row of P contains the (x, y) or (x, y, z) coordinates of a sample point. The sample points should be unique. However, if the sample points contain duplicates, scatteredInterpolant displays a warning and merges the duplicates into a single point.

Data Types: double

v — Sample values

vector

Sample values, specified as a vector that defines the function values at the sample points, v = F(x,y,z).

Data Types: double

Answer 5

[Mark Brandon]

SINGLE PRECISON REVISITED

I am writing with a follow up to my original question. I have now finished a bunch of testing with the program described above. I am using this program on a high-performance cluster, using a single node with 28 cpus and 247 Gb of dynamically allocated memory. I run this program simultaneous across as many cpus as possible. I have to limit the number of active cpus if the program uses too much memory, so there is a tradeoff between memory and time.

With my experimentation, I did determine that my computation did not require double precision. (That said, I do appreciate John D'Errico's caution about the "numerical cliff".)

However, I also found that the single precision DID NOT help with speed or memory use.

Walter Roberson's comment above helps put this result into perspective. It is my understanding (based on experience and also additional reading) that there is rarely a speed advantage for single precision relative to double precision. My sense is that modern 64-bit cpus are highly optimized for double-precision calculations, and Matlab has no doubt added to that optimization.

That said, my main interest was to be able to use less memory (which would allow me to use more of the cpus on the node). However, my peak memory use (Real Memory) did not change for single precision runs relative to double precision runs. This was surprising but I came to realize that the bottle neck is probably the scatteredInterpolant method, which requires all input and output arguments to be in double precision. The temporary conversion of arrays and vectors from single to double precision creates a recurring demand for lots more Real Memory. My guess is scatteredInterpolant is optimized for double precison. It is remarkably fast for the task it does!!

By the way, I want to give a call out for a very nice python program, mem_usage_ui-master, which provides a running profile of real memory use for Unix and mac OS machines. As some might know, Matlab's profile function only provides memory information for Windows operating systems. Check it out: https://pypi.org/project/mem-usage-ui/ This widget really helped me see real time how memory was being used. Perhaps Matlab could add this functionality...

My closing recommendation: Stick with the default double precision option. Matlab is very nicely optimized for double precision!!

But keep in mind that there might be cases where single precision could be useful, especially when memory is at a premium.

Note, however, that there are still a small set of Matlab functions that don't support single precision. Why?? I mentioned above the scatteredInterpolant method, which might be highly dependent on double precision. But consider two other functions I found--text and contourc--that work only with double precison. I have no idea why they cannot be expanded to use single precision...

Best,

Mark