# Bootstrap for surface fit goodness of fit stats

3 views (last 30 days)
Juan Leon on 4 May 2011
I am trying to use bootstrap on the MSE, R-squared output generated from a surface fit. However, I am having trouble figuring out how to set it up with the correct arguments.
I tried to do something like:
[fitresult1000,gof1000] = bootstrp(1000,'fit',x,y,z,'poly22',opts);
but I always get the following error message
Unable to evaluate BOOTFUN with the supplied arguments.
I figure this should be pretty simple, so I don't know what I am missing, if I can't figure this out I will probably just hand code it in Python, however it seems like a waste of time if Matlab has a simple function that will do it.
Anyway, any input you could give me would be appreciated.

Richard Willey on 12 May 2011
Few quick observations
First: There are a lot of different ways to bootstrap a regression model. Some of the more common examples include
Parametric residual bootstrap:
• Start by generating a regression model
• Calculate the variance of the residuals
• Using this information to generate new residuals
• Generate new observations by adding new residuals to YHat
Nonparametric residual bootstrap
• Start by generating a regression model
• Calculate the residuals
• Generate new observations by sampling with replacement from the set of residuals and adding these to YHat
Paired bootstrap
• Sample with replacement from the set of observations
Each of these techniques has its own advantages and disadvantages. (At the most basic level, there is a tradeoff between convergence speed and how restrictive your assumptions are about the data set) For basic information about bootstrapping regression models, I strongly recommend the following
Second: There is also the whole issue about how best to get this up and running in MATLAB
Here’s some simple code that will show you how to bootstrap SSE and R^2 using a parametric residual bootstrap. You can easily change this to a nonparametric residual bootstrap using the randsample function to sample with replacement.
X1 = 100 * rand(100,1);
X2 = 100 * rand(100,1);
Y = 2*X1 + 3*X2 + 10*randn(100,1);
[foo GoF] = fit([X1 X2],Y, 'poly11')
YHat = foo(X1,X2);
resid = Y - YHat;
Scaling = std(resid);
% Simple bootstrap example
N_Boot = 1000;
SSE = zeros(N_Boot,1);
R_Sqrd = zeros(N_Boot,1);
for i = 1:N_Boot
[foo_b , GoF_b] = fit([X1 X2],YHat + Scaling * randn(100,1), 'poly11');
SSE(i) = GoF_b.sse;
R_Sqrd(i) = GoF_b.rsquare;
end
mean(SSE)
std(SSE)
mean(R_Sqrd)
std(R_Sqrd)

Walter Roberson on 4 May 2011
The documentation for bootstrp indicates (at least in the current version) that,
bootfun is a function handle specified with @.
You have instead passed a string.

Juan Leon on 4 May 2011
I still get the same error message when I use @fit instead of 'fit'.
I guess I am not sure how I am supposed to pass in the BOOTFUN which in this case is:
fit([x,y,], z,'poly22',opts);
from the documentation, the input arguments passed into bootstrp are scalars, column vectors, or matrices, so I don't know how the 'poly22' and opts would be passed in.

Drew Steen on 12 May 2011
Hard to tell without knowing the details of the function 'fit' you're using, but I think the issue is that bootstrp will only accept numerical data as arguements. I believe the string 'poly22' is throwing it off.