Error in filtering data

I am trying to filter sla data which has 2191 data points (daily data from 2013 to 2018). The code I am using is given below. While I run this code, I am getting an error.

In filtfilt>getCoeffsAndInitialConditions (line 200)

In filtfilt (line 97)

In filter_matlab (line 8)

Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 1.419047e-16.

What is the meaning of it?

My code is:

sla = ncread('sla_ave.nc', 'SLA_AVE');

highcut = (2191/100)/(2191/2); %define highcut frequency 100 days in radians

lowcut =(2191/250)/(2191/2); %define lowcut frequency 250 days in radians

[b,a] = butter(6,[lowcut highcut]); %create 6th order butterworth band pass filter

freqz(b,a); %look at your filter characteristics

FilteredData = filtfilt(b,a,sla); %filter Data without phase shift

t=datenum(datestr(ncread('sla_ave.nc', 'TIME')+datenum(1950,1,1)));

plot(t, sla, 'linewidth', 1.5);

hold on;

plot(t, FilteredData, 'linewidth', 1.5);

axis tight;

xtck=t(1:100:2191);

set(gca,'XTick', xtck);

set(gca,'XTickLabel',datestr(t(1:100:2191),'m-yy'),'FontWeight','bold','FontSize',8);

Also, What is the meaning of defining 100 days in radians. Why is it necessary?

Answer 1

[Anudeep Kumar]

Hey Trishneeta,

I believe "matrix is close to singular or badly scaled" warning in MATLAB, typically indicates numerical instability in the filter design or application.

The error is thrown in the 'filtfilt' function, which applies a zero-phase digital filter. This usually happens when:

1. Filter coefficients '(b, a)' are poorly conditioned, often due to extreme or invalid cutoff frequencies.
2. The data has NaNs or Infs, which can break the filtering process.
3. The filter band is too narrow, especially for high-order filters.

Lookin closely in the code:

[b,a] = butter(6,[lowcut highcut]);

It is defined as:

highcut = (2191/100)/(2191/2); % which equals to 0.02
lowcut  = (2191/250)/(2191/2); % which equals to 0.008

So the bandpass is '[0.008 0.02]', which is very narrow and close to 0. This can cause instability in a 6th-order Butterworth filter.

To rectify this issue we can

• Try reducing the filter order (e.g., from 6 to 4).
• Check for NaNs in 'sla' using 'any(isnan(sla))'.
• Normalize the data if it has very large or very small values.

For the second question,

In digital signal processing, filter design functions like 'butter' expects normalized frequencies:

• The Nyquist frequency is half the sampling rate.
• So, normalized frequency = (desired frequency) / (Nyquist frequency).

In this case:

• There is daily data 2191, so 1 sample/day.
• Nyquist frequency = 2191/2 cycles/day.
• A 100-day period corresponds to a frequency of 2191/100
• Normalized frequency = (2191/100)/ (2191/2) = 0.02.

So according to my understanding we are normalizing the frequency relative to the Nyquist frequency.

I hope it helps!