How to reduce precision

1 view (last 30 days)
F?rat
F?rat on 27 Dec 2011
Hello, I am using cross products to find an angle between two vectors (not using dot product because I am also using the cross product resultant vector) and in the limits, I get numbers such as e^-15 which I do not need. I need at most 4 digits after the point. Is there a code or simple way other than dividing with 0.001, rounding and mltiplying again with 0.001. Any help would be appreciated. F?rat

Answers (1)

Walter Roberson
Walter Roberson on 28 Dec 2011
Multiplying by 10000, rounding, and dividing by 10000 is about the best possible in MATLAB.
Warning: Except in the cases that are integer multiples of exact powers of 2, such as 0.75 (3 * 2^(-2)), it is not possible in MATLAB to truncate floating point numbers to have exactly a certain number of decimal places. Floating point numbers are represented in binary, and none of 0.1 and 0.01 and 0.001 and 0.0001 have exact binary representations. You will end up with low-order bits set so as to come as close as possible to a multiple of 0.0001 in binary representation, but you will never get 0.0001 exactly.
Try
fprint('%.99g\n', 0.0001)
and you will see that the value is not exactly 0.0001
  1 Comment
Walter Roberson
Walter Roberson on 28 Dec 2011
Note: the Symbolic Toolbox can do decimal arithmetic, as can the Fixed Point Toolbox.

Sign in to comment.

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!