MATLAB Answers

Bootstrap for surface fit goodness of fit stats

11 views (last 30 days)
Juan Leon
Juan Leon on 4 May 2011
I am trying to use bootstrap on the MSE, R-squared output generated from a surface fit. However, I am having trouble figuring out how to set it up with the correct arguments.
I tried to do something like:
[fitresult1000,gof1000] = bootstrp(1000,'fit',x,y,z,'poly22',opts);
but I always get the following error message
Unable to evaluate BOOTFUN with the supplied arguments.
I figure this should be pretty simple, so I don't know what I am missing, if I can't figure this out I will probably just hand code it in Python, however it seems like a waste of time if Matlab has a simple function that will do it.
Anyway, any input you could give me would be appreciated.

Answers (4)

Richard Willey
Richard Willey on 12 May 2011
Few quick observations
First: There are a lot of different ways to bootstrap a regression model. Some of the more common examples include
Parametric residual bootstrap:
  • Start by generating a regression model
  • Calculate the variance of the residuals
  • Using this information to generate new residuals
  • Generate new observations by adding new residuals to YHat
Nonparametric residual bootstrap
  • Start by generating a regression model
  • Calculate the residuals
  • Generate new observations by sampling with replacement from the set of residuals and adding these to YHat
Paired bootstrap
  • Sample with replacement from the set of observations
Each of these techniques has its own advantages and disadvantages. (At the most basic level, there is a tradeoff between convergence speed and how restrictive your assumptions are about the data set) For basic information about bootstrapping regression models, I strongly recommend the following
Second: There is also the whole issue about how best to get this up and running in MATLAB
Here’s some simple code that will show you how to bootstrap SSE and R^2 using a parametric residual bootstrap. You can easily change this to a nonparametric residual bootstrap using the randsample function to sample with replacement.
X1 = 100 * rand(100,1);
X2 = 100 * rand(100,1);
Y = 2*X1 + 3*X2 + 10*randn(100,1);
[foo GoF] = fit([X1 X2],Y, 'poly11')
YHat = foo(X1,X2);
resid = Y - YHat;
Scaling = std(resid);
% Simple bootstrap example
N_Boot = 1000;
SSE = zeros(N_Boot,1);
R_Sqrd = zeros(N_Boot,1);
for i = 1:N_Boot
[foo_b , GoF_b] = fit([X1 X2],YHat + Scaling * randn(100,1), 'poly11');
SSE(i) = GoF_b.sse;
R_Sqrd(i) = GoF_b.rsquare;
end
mean(SSE)
std(SSE)
mean(R_Sqrd)
std(R_Sqrd)

Walter Roberson
Walter Roberson on 4 May 2011
The documentation for bootstrp indicates (at least in the current version) that,
bootfun is a function handle specified with @.
You have instead passed a string.

Juan Leon
Juan Leon on 4 May 2011
I still get the same error message when I use @fit instead of 'fit'.
I guess I am not sure how I am supposed to pass in the BOOTFUN which in this case is:
fit([x,y,], z,'poly22',opts);
from the documentation, the input arguments passed into bootstrp are scalars, column vectors, or matrices, so I don't know how the 'poly22' and opts would be passed in.

Drew Steen
Drew Steen on 12 May 2011
Hard to tell without knowing the details of the function 'fit' you're using, but I think the issue is that bootstrp will only accept numerical data as arguements. I believe the string 'poly22' is throwing it off.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!