Fit coefficients are equal (to starting points) in 3000 different fits

Hey all,

I'm trying to get the fit coefficients data, for about 3000 fits with added data, starting with 10 points and adding another and another until I get to 2999 points.

My goal is to see how the fit coefficients, and eventually the gooddness of fit as well, changes over the change in the data. When I run my script the coefficients end up fixed as their start points. What did I do wrong?

Another question is do you maybe know how do I extract the R-square value out of the fit in a similar way? and the coefficients errors?

One last less urgent question - right now I need to fill manually the data array (x,y as follows) and cannot extract it from the workspace. Is there a way to use the workspace here?

I'm adding my script for clearer understanding.

Thanks in advance!

function[x,y,f] = GraphFittingAnalysisFrom0 (x,y,f) % x,y are columns and f the fitting function.

prompt_x = ['x='];

prompt_y = ['y='];

prompt_f = ['fit type f='];

x = input(prompt_x);

y = input(prompt_y);

f=input(prompt_f);

Fit = fit(x,y,f);

A = coeffnames(Fit);

Num_of_coeff = size(A,1);

disp(['the number of coefficients is ', num2str(Num_of_coeff)]) % check by output

Start_point = zeros([Num_of_coeff 1]); % asking user to enter coefficients' start point

for i = 1: Num_of_coeff

Start_point(i) = input(['start point of ',A{i},' ']);

end

Coefficients = zeros([Num_of_coeff 2999]); % where values of coefficients will be saved

for n = 10:2999 % the problem is in the fits here -it does not change the coefficients and give them as their start points

x_n = x(1:n);

y_n = y(1:n);

options = fitoptions(f);

options.Robust = 'LAR';

options.StartPoint = Start_point;

options.MaxIter = 25000;

options.TolFun = 25000;

Fit_n = fit(x_n,y_n,f,options);

Val = coeffvalues(Fit_n);

Coefficients(:,n) = Val.'; % saving the coefficients values of the nth fit in the nth column of Coefficeints

end

disp (Coefficients)

tiledlayout(1,Num_of_coeff) % plotting the coefficients graphs

X=[1:2999];

for i = 1 : Num_of_coeff

Y=Coefficients(i,:);

nexttile

plot(X,Y)

title(A{i})

end

end

[Matt J]

You need to show an example of a fit that failed.

[Gil]

Hey @Matt J, thanks for the answer.

It's not really one fit that failed. It's 2989 fits all with the same result (coefficients equal to start point).

It happens with any data, function and start points I try.

I've edited the title of the question to maybe get better understanding.

[Torsten]

@Gil

x = input(prompt_x);
y = input(prompt_y);

And the user inputs 2999 points for x and y :-) ?

Or do you simply overwrite here the inputs x,y and f to the function "GraphFittingAnalysisFrom0" ?

For a first test, you should hardcode all the inputs for the optimizer.

[Matt J]

It happens with any data, function and start points I try.

You need to give us the means of duplicating what you have tried. We don't know what specific functions and input you yourself have tested.

[Gil]

@Matt J @Torsten

I take x and y to be column vectors of size 2999, for example take x to be all integers between 1 and 2999, and y some experiment results (attached to this comment), then I insert a function f for the fit, for example 'a.*x.^b+c' (you need to insert it with the apostrophes). Now I insert my start points, based on calculations I've made prehand (a=-0.1017, b=-0.04284, c=0.105).

It's supposed to make the fit for all vectors x_n,y_n which are the first n points of x and y I inserted and then save their coefficients values in the nth column of a matrix I called "Coefficients" (which is the size of the number of coeffcients x2999).

The problem is that all the fits end up with coefficients equal to their start points.

1. I ran it without the "problematic for loop" (with same variables) for some random n (say n=250) and got the same results as before (a,b,c equal to their start point for this fit).
2. when I tried it for x=y, f=ax+b and unresonable start points (a starts at 0 and b starts at 1), I get the expected fit (a=1 and b=0) for any n larger then 25.
3. Doing these fits manually using cftool with the same fit options, gives me good results (specifically the coefficients are different from their start points and differ by the number of excluded points at the end).
4. Edit: just to convince you, I attach the plots of the fit coefficients for number of points in the fit.

Thanks a lot!

[Cris LaPierre]

The concept of what you are trying to do is easy enough to understand, but I'm also lost at what data you are fitting to. For example, the longest timeseries in the attached timeseries data has 2001 data points, so how/why do you use 2999?

Could you please share how you create the function inputs x,y, and f using the data you have shared? Please correct/update my attempt below. Code has been modified to at least run here.

load XYData.mat

x = squeeze(expX.Data);
y = squeeze(expY.Data);
f = 'a.*x.^b+c';

[x,y,f] = GraphFittingAnalysisFrom0 (x,y,f)

Warning: Start point not provided, choosing random start point.

the number of coefficients is 3

Index exceeds the number of array elements. Index must not exceed 21.

Error in solution>GraphFittingAnalysisFrom0 (line 27)
x_n = x(1:n);

function[x,y,f] = GraphFittingAnalysisFrom0 (x,y,f)  % x,y are columns and f the fitting function.
% prompt_x = ['x='];  
% prompt_y = ['y='];
% prompt_f = ['fit type f='];
% x = input(prompt_x);
% y = input(prompt_y);
% f=input(prompt_f);
Fit = fit(x,y,f);
A = coeffnames(Fit);
Num_of_coeff = size(A,1);
disp(['the number of coefficients is ', num2str(Num_of_coeff)]) % check by output
Start_point = zeros([Num_of_coeff 1]);  % asking user to enter coefficients' start point
% for i = 1: Num_of_coeff   
% Start_point(i) = input(['start point of ',A{i},'  ']);
% end
Start_point = [-0.1017;-0.04284;0.105];
Coefficients = zeros([Num_of_coeff 2999]);  % where values of coefficients will be saved
for n = 10:2999     % the problem is in the fits here -it does not change the coefficients and give them as their start points
	x_n = x(1:n);
	y_n = y(1:n);
	options = fitoptions(f);
	options.Robust = 'LAR';
	options.StartPoint = Start_point;
	options.MaxIter = 25000;
	options.TolFun = 25000;
	Fit_n = fit(x_n,y_n,f,options);
	Val = coeffvalues(Fit_n);
	Coefficients(:,n) = Val.';   % saving the coefficients values of the nth fit in the nth column of Coefficeints
end
disp (Coefficients)
tiledlayout(1,Num_of_coeff) % plotting the coefficients graphs
X=[1:2999];
for i = 1 : Num_of_coeff 
    Y=Coefficients(i,:);
    nexttile
    plot(X,Y)
    title(A{i})
end
end

[Gil]

Hey @Cris LaPierre,

sorry, but I don't really understand what you mean. Both vectors I've uploaded have 2999 points.

[Cris LaPierre]

My apologies. It looks like I was accidentally loading the wrong mat file on my computer (XYData instead of xyvectors)

Answer 1

[Bruno Luong]

Can you do check whereas they are identical with this command after the loop

isequal(Coefficients(:,10),Coefficients(:,2999))
fprintf('%e ', Coefficients(:,10)-Coefficients(:,2999))
fprintf('\n')

[Gil]

@Bruno Luong they are

[Bruno Luong]

Are you sure is this huge value?

options.TolFun = 25000

The starting point probably satisties such lostly tolerance so the fit give you back what you feed in.

(PS also; Highly recommended you check correctness of output.exitflag with

[Fit_n, ~, output] = fit(x_n,y_n,f,options);

)

[Gil]

@Bruno Luong. Of course you're right! Thanks!!!!