Problem passing function to sequentialfs

Hi.

I am trying to select the variables that best predict the outcome of a patient group using sequentialfs and the AUC of a model that combines the variables. This is my code:

 function inmodel=call_sequentialfs(x,y)
    c = cvpartition(length(y),'LeaveOut');
    inmodel = sequentialfs(@compAUC2,x,y,'cv',c);

    function [AUC]=compAUC2(x,y)
        clas = fitcdiscr(x,y,'DiscrimType','linear');
        [~,RSvar] = resubPredict(clas);
        [~,~,~,AUC] = perfcurve(y,RSvar(:,2),1);
    end 
end

I get the following error:

Error using crossval>evalFun (line 480)
The function 'call_sequentialfs/compAUC2' generated the following error:
Too many input arguments.

I get no errors when I call

compAUC2(x,y)

from the command line. Digging into the crosseval code, it seems to expect the function compAUC2 to have 4 arguments... Why is that? Can you help?

Many thanks,

Marta

[Utkarsh Singh]

The code goes like this: (refer to sequentialfs)

load fisheriris;
X = randn(150,10);
X(:,[1 3 5 7 ])= meas;
y = species;
c = cvpartition(y,'k',10);
opts = statset('display','iter');
fun = @(XT,yT,Xt,yt)...
      (sum(~strcmp(yt,classify(Xt,XT,yT,'quadratic'))));
[fs,history] = sequentialfs(fun,X,y,'cv',c,'options',opts)

Following error is being generated when i run the code:

Start forward sequential feature selection:
Initial columns included:  none
Columns that can not be included:  none
Error using crossval>evalFun (line 480)
The function '@(XT,yT,Xt,yt)(sum(~strcmp(yt,classify(Xt,XT,yT,'quadratic'))))'
generated the following error:
Too many input arguments.
Error in crossval>getFuncVal (line 497)
funResult = evalFun(funorStr,arg(:));
Error in crossval (line 343)
    funResult = getFuncVal(1, nData, cvp, data, funorStr, []);
Error in sequentialfs>callfun (line 485)
    funResult = crossval(fun,x,other_data{:},...

Someone please help me, thanks in advance!

Answer 1

[Michael Wang]

sequentialfs performs “c”-fold cross-validation by repeatedly calling fun with different training subsets of X and y, XTRAIN and ytrain, and test subsets of X and y,XTEST and ytest.

http://www.mathworks.com/help/releases/R2016a/stats/sequentialfs.html

Therefore, it needs four inputs. In your case XTRAIN, YTRAIN, XTEST and YTEST

Here is an example:

function inmodel=call_sequentialfs(x,y)
        c = cvpartition(length(y),'LeaveOut');
        inmodel = sequentialfs(@compAUC2,x,y,'cv',c);
        function [AUC]=compAUC2(XTrain,YTrian,XTest,YTest)
           AUC = (sum(~strcmp(YTest,classify(XTest,XTrain,YTrian,'quadratic')))); 
        end 
    end

Hope this helps

[Javier Imaz Higuera]

How could you add the type function that classify is going to use? I want it to be an input function from my MAIN function.

function inmodel=call_sequentialfs(x,y,typeFunc)
        c = cvpartition(length(y),'LeaveOut');
        inmodel = sequentialfs(@compAUC2,x,y,'cv',c); % how to pass typeFunc to the function??
    function [AUC]=compAUC2(XTrain,YTrian,XTest,YTest,typeFunc)
           AUC = (sum(~strcmp(YTest,classify(XTest,XTrain,YTrian,typeFunc)))); 
        end 
    end