Using "find" for finding decimal values

Hi

I use the matlab command importdata:

X = importdata('filename.csv');

to read in a csv file with three columns.

Now, for finding a specific values in the matrix X, I simple use the find command as follows:

idx = find(X(:,1 ) == 17)

But, the same seems not to be possible for decimal numbers. This for instance would not work:

idx = find(X(:,1 ) == 17.9203)

even though 17.9203 is to be found in the original csv file. What is the problem and what can I do?

Thanks

Answer 1

[Matt Fig]

MiauMiau, the numbers you are looking for are obviously different from the short format you see. Do this:

[~,idx] = min(abs(X(:,1) - 6.0018));
Y = X(idx,1);
[~,idx] = min(abs(X(:,1) - 17.9203));
fprintf('%15.15f   %15.15f\n',Y,X(idx,1))

And show us the ouput in a comment on this answer, don't add another answer!

I have a feeling you will need to set your tolerance much higher, like 10^-4.

[MiauMiau]

I am sorry, I am totally new here, will listen to your advices though! Please be a bit patient with me at the time being :-)

Ok, so, I have used your suggestion, and this is what I get then:

>> ratiop

    6.001772842630019   17.876593908536755

[Matt Fig]

So it is as I thought. You need to use a much higher tolerance than eps(N). Though I am confused as to why you thought there was a number like 17.9203 when the closest number to 17.9203 is 17.8766. Why did you think there was a 17.9203?

Anyway, it would seem you should use:

idx = find(abs(X(:,1 ) - 6.0018) < 10^-3)

You could always use the method I gave above to find the closes number to a desired number, just like you found the closest number to 17.9203 is 17.8766.

[MiauMiau]

Thanks - it works perfectly, wow! I thought there is a number like 17.9203 because I used two input files which were supposed to be identical - obviously there were not. May I ask how you found out the tolerance has to be set to 10^-3? I don't understand that yet. Thank you very much!

[Matt Fig]

I found out by looking at how close your guess value was to the actual value. You may have to do some tweaking but you get the idea now.

Answer 2

[Azzi Abdelmalek]

Try

 idx = find(abs(X(:,1 )-17.9203)<eps)

[Jan]

Substracting two numbers in the magnitude of 20 can have a roundoff error of eps(20). Then this is better:

idx = find(abs(X(:,1 ) - 17.9203) <= eps(17.9203))  % EDITED: was "<"

Even 10*eps(17.9203) would be a reliable limit.

[EDITED] Thanks, Kye. I cannot test this currently. But I assume that even "<=" can fail, when the values are at the limits of 2^n. Does eps(16 + eps(16)) reply eps(32)? Then a factor of 2 would be obligatory.

[Kye Taylor]

should be <=

Answer 3

[Jan]

There is no exact representation of decimal floating point numbers in binary format for all values. You find a lot of corresponding discussion in this forum:

0.1 + 0.2 - 0.3 == 0
>> false

This is no bug, but the expected behaviour, when floating point numbers are represented with a limited precision.

A consequence is, that you cannot compare numbers like 17.9203 and 17.92029999999999999999 sufficiently and even the display in the command window can be rather confusing.

Answer 4

[MiauMiau]

I was not aware of that, that is pretty horrible! Anyway, both of your suggestions did not work, so also for:

idx = find(abs(X(:,1 ) - 17.9203) < eps(17.9203))

I did get an empty matrix returned - although the value is to be found in X. Also, when I changed the number a bit, I then got too many answers returned. But what strikes me most, is that in the first case, I just did not get any answer at all!..?!

[Azzi Abdelmalek]

What do you get when you type

 min(abs(X(:)-17.9203))

[MiauMiau]

with:

idx = find(min(abs(X(:,1)-12.8487)))

I get always idx = 1

with only:

idx = min(abs(X(:,1)-12.8487))

I get some very small number for which I have no idea where it does come from...

[Jan]

Please do not post "a very small number" but the actual value. And the source of this small number has been explained already.

Differences between two numbers a and b can have a round-off error of eps(max(a, b)). Therefore checking < eps(17.9203) is not sufficient, because you need at least "<=", but an additional factor of 2 or 16 is more likely to match your needs reliably.

These effects are not horrible. They should be explained exhaustively in the very first lesson for numerical computations. Limited accuracy is not only an effect in applied numerics but you find this in the nature of physical effects also. The number of physical effects which are well defined for more than 15 significant digits is extremely small.

[Azzi Abdelmalek]

It's obvious to find find(min(abs(X(:,1)-12.8487))) equal to 1, because min(abs(X(:,1)-12.8487)) is a scalar. Finding a very smal number, means that the number 12.8487 exists in your matrix, and for problems caused by how matlab stores numbers, you can't get, sometimes, the exact equality.

[Jan]

@Azzi: This is not a problem of Matlab. The IEEE-754 standard for the type double is well established and used in almost all numerical programming environments, e.g. it is hard coded in the processor architectures except for a small number of exceptions (I assume Walter knows some). You will find exactly the same effects, when you program this in C, FORTRAN, Haskel, VisualBasic or directly in machine code.

[Azzi Abdelmalek]

Yes, I said Matlab because we are working with Matlab. I think, representing real numbers for numerical programming is almost similar to what an ADC (analogic-digital converter) do. There is a quantification of a real number, which means there is an error of quantification which depends on the number of used bits and method.

[Jan]

@Azzi: I've stressed this detail, because the OP seems to be confused about this topic already. I did not assume, that you struggle with floating point arithmetics.

Answer 5

[MiauMiau]

I am really sorry for my use of the word "horrible".

If you meant the following command though ( I have used another nummerical value):

idx = find(abs(X(:,1 ) - 6.0018) <= eps(6.0018))

I - again - got an empty 0,1-matrix.

Also, I was not criticising the obviously necessary restriction on measurement and computation precision but I was amazed that there is not a simpler command to handle that. Anyway. What shall I try next?

[Jan]

Why do you assume that 6.0018 is an element of X?