How do I differentiate which pixels are classified by the LDA?

Code to activate LDA:

data = importdata('LDA data.mat')

features=data(:,1:end-1); %split data without labels

lable=data(:,end); %get the labels

W=LDA(features,lable); %perform LDA on data

L = [ones(170884,1) features] * W';

P = exp(L) ./ repmat(sum(exp(L),1),[170884 1]);

handles.features = features;

guidata(hObject, handles);

========================================================================

LDA code:

% LDA - MATLAB subroutine to perform linear discriminant analysis

% by Will Dwinnell and Deniz Sevis
%
% Use:

% W = LDA(Input,Target,Priors)

%
% W       = discovered linear coefficients (first column is the constants)

% Input   = predictor data (variables in columns, observations in rows)

% Target  = target variable (class labels)

% Priors  = vector of prior probabilities (optional)
%
% Note: discriminant coefficients are stored in W in the order of unique

(Target)
%
% Example:
%
% % Generate example data: 2 groups, of 10 and 15, respectively

% X = [randn(10,2); randn(15,2) + 1.5];  Y = [zeros(10,1); ones(15,1)];
%
% % Calculate linear discriminant coefficients

% W = LDA(X,Y);
%
% % Calulcate linear scores for training data

% L = [ones(25,1) X] * W';
%
% % Calculate class probabilities

% P = exp(L) ./ repmat(sum(exp(L),2),[1 2]);
%
%
% Last modified: Dec-11-2010

function W = LDA(Input,Target,Priors)

% Determine size of input data

[n m] = size(Input);

% Discover and count unique class labels

ClassLabel = unique(Target);

k = length(ClassLabel);

% Initialize

nGroup     = NaN(k,1);     % Group counts

GroupMean  = NaN(k,m);     % Group sample means

PooledCov  = zeros(m,m);   % Pooled covariance

W          = NaN(k,m+1);   % model coefficients

if  (nargin >= 3)  PriorProb = Priors;  end

% Loop over classes to perform intermediate calculations

for i = 1:k,

      % Establish location and size of each class

      Group      = (Target == ClassLabel(i));

      nGroup(i)  = sum(double(Group));

      % Calculate group mean vectors

      GroupMean(i,:) = mean(Input(Group,:));

      % Accumulate pooled covariance information

      PooledCov = PooledCov + ((nGroup(i) - 1) / (n - k) ).* cov(Input(Group,:));

end

% Assign prior probabilities

if  (nargin >= 3)

      % Use the user-supplied priors

      PriorProb = Priors;

else

      % Use the sample probabilities

      PriorProb = nGroup / n;

end

% Loop over classes to calculate linear discriminant coefficients

for i = 1:k,

      % Intermediate calculation for efficiency

      % This replaces:  GroupMean(g,:) * inv(PooledCov)

      Temp = GroupMean(i,:) / PooledCov;

      % Constant

      W(i,1) = -0.5 * Temp * GroupMean(i,:)' + log(PriorProb(i));

      % Linear

      W(i,2:end) = Temp;

end

% Housekeeping

clear Temp

end

% EOF

Here's the catch, when I add all the P's together, it dosen't give 1 instead it gives exponential values that are to the power of negative 100+. When I remove the exponentials, the values are still too small. Can anyone point out what's wrong? How do I get labels from this?

[Walter Roberson]

Is there a difference between this question and http://www.mathworks.co.uk/matlabcentral/answers/60067-lda-showed-probability-problems-in-calculating-probability-of-labels ?

[Lester Lim]

Apologise for the multiple postings of similar questions, desparate to get the supervised learning done cause deadline nearing. Seek your understanding...

Answer 1

[Walter Roberson]

If I trace correctly, L has multiple rows and columns. sum() of an array with multiple rows and columns defaults to summing the columns. I suspect you want to sum the rows. That would be sum(L,2)

[Lester Lim]

Yup, L has multiple rows and columns. Yup, when I sum the rows, it has to be one but its 10 power of negative 100+ which is not right.

[Lester Lim]

I tried, its much better but still not right, because they don't add up to one and all are still 10 power of -6... Any more good suggestions?

[Walter Roberson]

I do not immediately see any reason why L should sum to 1 ?

[Lester Lim]

Aside from the maximum variable size allowed space issue, it should sum up to 1 because 1/100. Like this it would be more obvious which of the 7 rows would get to be labelled one. My bad if I forgot to mention, LDA data is made of 21 textured pictures derived from the same image, reshaped into a column. There is a 22nd column used to make the labels.

[Walter Roberson]

What is 1/100 ?

Perhaps you should put in a breakpoint and check the return value, W, from your LDA routine.

[Lester Lim]

The 1/100 is just to express te values as a percentage of 1. Example, when P is caluclated, the seven values should add up to 1.

[Walter Roberson]

Please put in a breakpoint and examine the values W, returned from your LDA routine.

[Lester Lim]

In W, I got these values in a 7x22 double format:

-193.491017461002  -5.50959267367006  111.708524412789  87.6218871936319
  446.700232373038  146.621291920479  -96.9097196894522  118.426774882096  50.0100103932259  121.378830671693  56.4864635582821  -2.76841062133764  -16.1120711740305  -397.488293163878  -77.1221723389481  133.251816119721  -7.66496370684530  18.9603293801712  15.2298595389594  4.69172416523919  -37.0196707061391  -58.9803292938609
-160.791499287099  -33.9212327700470  112.181397382687  62.3587240535485  404.703667859125  98.8675640648398  -85.7167187999254  113.585338285779  65.0913301901188  112.902327360163  41.4274113199181  -41.8561740099175  28.8803959784345  -359.263616645855  -39.8911719350572  120.138000661510  -4.13894549727915  19.4364725936685  21.2377495255429  35.9884615044051  -37.9051929802253  -90.0948070197747
-192.637958685202  -9.63127598602940  96.3934528194520  89.5312430613559  442.588049746156  133.698586989052  -85.0226581155444  108.511382424977  51.2977203158707  140.301938158494  70.4868645348855  -12.0771022000935  -21.7009708975626  -392.437408779460  -67.2131774147138  117.101151432058  -1.77530612230235  12.7041555625108  15.7524175890331  43.1012947829630  -18.7328950016230  -109.267104998377
-202.595050316010  -1.55376496254347  118.649759336485  93.8930626639790  434.739255650713  104.956551494712  -97.1711578483496  121.135065496462  39.2683647144766  136.979338410992  53.5687077185500  5.97040028388890  -26.2698724742241  -387.755531512514  -40.2807465517246  130.028299742292  -12.3406726984797  21.2792851213477  20.1048755650344  29.0257967749535  -39.8643123589770  -88.1356876410230
-216.060378083838  16.9116906277240  112.923865960571  71.1389041535390  452.433161069958  174.589915493961  -81.4639686392297  129.785781344264  70.5884264505651  113.170453788147  54.9491936001823  -20.3899128679517  -1.83495201229514  -404.167612781858  -103.040346911146  120.784105919671  -4.50233320004713  17.5426752561411  16.1574237364482  27.7994652040965  -45.4383627187752  -82.5616372812248
-229.581478003519  4.07906124084546  101.809812676786  92.3233333630989  505.001796939196  186.094898592469  -116.166935851327  124.869704069559  78.6773757156870  124.364363108760  52.7776890047124  -16.9317416156550  -0.990576781157522  -452.425878446712  -115.990848516579  151.569482008695  -6.68716352903964  19.1164167395201  11.2078760922383  18.3165325763379  -41.3743318705245  -54.6256681294755
-231.895925708761  -5.84506511166657  139.668295649444  85.0593949233774  490.959274155139  302.135979786930  -43.8913189184604  131.873980499463  70.8330535348486  114.857846114061  50.4627076871159  -23.2241774830022  4.70478615001664  -442.765453305376  -223.882294077085  80.7659303972797  -8.38243820615552  21.4109005463525  15.1760092359402  43.4213123511506  -51.5251175205718  -108.474882479428

[Walter Roberson]

Is "features" 170884 by 6?

I do not see any reason, from those values, to expect that L should sum to any particular value ?

[Lester Lim]

Its a 170884 by 21 double. L is 170884 by 7 double. But the main point is that I can't tell which probability has a higher chance of getting the label...