How to apply PCA and LDA to a data set with 3 subjects and 10features?

The problem is like this: I have a data set of 3 subjects, and for each subject has 6 samples, and each sample has 10 feature. According to my understanding, each feature can be treated as one dimension. So first, I apply PCA to do dimension reduction, since there got so many dimensions, I then choose to leave 3 principal component as xyz axis for the new system. The new coordinates system is easy to plot and for understanding.

But can anyone tell me how to do LDA after I get PCA data? all the theory I learnt is about 2 dimension. My problem is that after PCA dimension reduction, there's still 3 dimension left in the new system. (3 dimension, 3 class, each class has 6 samples)

Can anyone explain to me how to use LDA now? related material and source code also welcomed.

Following is the example data.

         *feature1   f2    f3    f4    f5    f6    f7    f8     f9   f10*

   sample1   13.0   15.1  37.5  52.9  30.0  74.0  53.8  102.7  22.9  48.9
        s2   12.7   15.4  37.3  52.9  29.4  76.7  55.3  107.1  23.5  51.8
subject s3   12.5   15.0  36.8  52.8  29.5  77.7  55.2  107.6  23.3  52.4
    1   s4   12.8   14.9  37.6  52.8  29.8  73.9  53.9  103.3  23.1  50.3
        ss   12.7   15.3  37.2  52.8  30.1  77.0  54.5  102.9  22.9  51.4
   sample6   12.9   15.3  36.9  52.8  29.4  76.5  54.9  105.2  22.7  52.3

   sample1   11.4   13.9  64.2  86.7  49.0  64.3  44.6  85.4   37.7  40.8
        s2   10.5   13.9  64.1  85.5  44.7  63.3  45.2  84.0   40.8  38.8
subject s3   11.3   14.0  63.3  80.5  46.9  63.1  44.0  84.1   35.8  40.4
    2   s4   11.4   13.8  62.2  82.2  49.4  63.9  44.4  84.8   31.1  40.1
        ss   10.2   13.5  63.1  86.2  45.7  63.3  44.8  84.2   40.9  39.8
   sample6   11.3   14.1  62.9  86.1  44.9  65.4  45.1  83.9   41.0  38.7

   sample1   14.3   17.5  54.0  68.5  36.0  86.8  64.2  116.0  30.5  51.8
        s2   14.1   17.0  54.2  63.9  36.2  83.8  60.5  112.5  30.4  51.4
subject s3   13.8   16.8  52.8  65.6  35.7  85.2  63.3  115.8  31.2  50.8
    3   s4   14.0   17.2  53.5  65.9  35.8  87.3  64.6  113.7  30.0  52.1
        ss   14.1   17.8  54.1  67.2  36.3  86.4  65.4  117.2  30.1  52.3
   sample6   14.2   16.5  53.6  68.1  36.8  86.8  64.7  116.6  30.5  51.1

VERY ERGENT! PLEASE HELP

Answer 1

[Hari]

Hi,

I understand that you have a dataset with 3 subjects, each having 6 samples and 10 features. You've applied PCA to reduce the data to 3 dimensions and now want to apply Linear Discriminant Analysis (LDA) to further distinguish between the subjects (classes).

I assume you want to use LDA as a supervised method for dimensionality reduction or classification after PCA, and you're looking for a conceptual approach without focusing on the MATLAB code.

In order to perform LDA on the PCA-reduced data, you can follow the below steps:

Step 1: Prepare your data and class labels

Organize your samples into a matrix where each row is a sample and each column is a feature. Also, create a label vector indicating the subject/class of each sample.

Step 2: Apply PCA to reduce dimensionality

Use PCA to transform your original 10-dimensional data into a new 3-dimensional space by keeping the top 3 principal components that capture the most variance.

Step 3: Apply LDA on the PCA-reduced data

Perform LDA using the 3D PCA output as input and the class labels to find the directions (discriminant axes) that best separate the 3 classes.

Step 4: Project the PCA-reduced data onto the LDA space

Transform the PCA-reduced data into the new LDA space. For 3 classes, LDA will generate up to 2 discriminant components, which you can then use for further analysis or visualization.

Step 5: Visualize or analyze the LDA result

You can now visualize the transformed data in the LDA space to inspect how well the classes are separated or use the LDA-transformed features for classification.

Refer to the documentation of PCA:

https://www.mathworks.com/help/stats/pca.html

Refer to the documentation of LDA using fitcdiscr:

https://www.mathworks.com/help/stats/fitcdiscr.html

Hope this helps!