Combines all the measurements into a single matrix

"I have limited scripting experience but understand that loops are powerful tools for automating repetitive tasks, and I want to use them to save time.

Currently, I need help writing a script that:

1. Imports data for multiple samples separately.
2. Combines all the measurements into a single matrix. For example:

• The first column should represent the wavelengths (constant for all samples).
• The remaining columns should represent the intensities for each sample.

The dataset size is 5826 rows by 156 columns. Could someone guide me on how to approach this using loops?"

[Mathieu NOE]

hello

you simply need to initialize a matrix and then fill the columns according to which data vector / array you have available.

we have no clue here how the "raw" data (wavelengths ,intensities ) is organized , so this is up to you - or please share your data (as mat file) if you want us to better help you

%  %The dataset size is 5826 rows by 156 columns
r = 5826; 
c = 156;
A = zeros(r,c); % or NaN if you prefer

% main loop
for k = 1:c
    A(:,k) = import_data(k) % or wahtever way to import the right column of data (need them !)
end

[Nadia]

Thankyou for your answer, its not working i am tryping to upload file but due to high mb its not possible. My data looks like this

188,008805 7 6,2 6,83333333333334 8,33333333333334

188,105527 9,2 6,4 9,5 8,33333333333334

188,202243 8,2 10,6 6,66666666666667 7,33333333333333

188,298953 8,6 7,2 7,16666666666667 7,66666666666667

188,395657 10 8,4 7 9

188,492355 8 8,4 7,66666666666667 10

188,589046 9,8 8,6 7,5 8,33333333333334

188,685731 7,4 6,8 8,66666666666667 7,66666666666667

188,78241 9,4 9,4 6,83333333333333 11,3333333333333

188,879083 5,8 10,4 7,66666666666667 7,33333333333333

188,975749 9,2 9,2 7,5 8,66666666666667

189,07241 9 7,8 7,66666666666667 7

I got response in 0000

[Stephen23]

"...i am tryping to upload file but due to high mb its not possible."

Do not upload the entire file. Modify and upload a subset of the filedata that accurately represents the file and data format.

Use a reliable text-editor to modify the file.

Do NOT use MS Excel to modify the file.

Do NOT overwrite your original data file.

[Nadia]

I have attached my file in total it,s 5826 rows by 156 columns. Total samples are 156

[Stephen23]

@Nadia: in contrast to what you showed in your earlier comment, the data file you uploaded has no decimal fractional values. Which one is correct?

The uploaded file format is rather odd: mostly tab-separated, but sometimes with random space chracters:

It is an odd mixture. The data you showed in your comment earlier used only tab delimiters.

Please check and if required upload file which accurately represents the data file format.

[Nadia]

I hope now it´s correct.

[Mathieu NOE]

hello again

a simple script below that will stack your data (assuming as you posted , the first column with wavelength is valid for all files)

to test my code I simply duplicated your Data.txt file in multiple copies of it , and so we are sure the data are processed in the way we want

not sure if that matters to you , but in case you need to proceed the files in some sorted way I recommend you use this Fex submission (excellent work !) , because dir only is not reliable in all circumstances

Natural-Order Filename Sort - File Exchange - MATLAB Central

if you need to add some processing of your data (I see you want to compute some means here) - we can esily adapt the code accordingly

%% define path
fileDir = pwd; % or your specific path / folder

S = dir(fullfile(fileDir,'Data*.txt')); % get list of data files in directory  

S = natsortfiles(S); % sort file names into natural order (what matlab dir does not well) , see FEX : 
           %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)
 
Nfiles = length(S);  % Number of files
combined_data= [];

%% Loop inside folder
for k = 1:length(S) % read data 
    fileName = S(k).name;
    data = readmatrix(fullfile(fileDir, fileName)); % or use a structure (S(k).data ) to store the full data structure
    
        % For the first file, store wavelengths in the first column
        if k == 1
            % store first time : Wavelengths + Intensities  
            combined_data = data; 
        else
             % next files
            combined_data= [combined_data data(:, 2:end)];  % then add Intensities only (skipp first column Wavelengths) for the next files

        end
        
end

[Mathieu NOE]

here some further code extension if you want to add a "mean intensity" column (appended at the end ) for each file -

%% define path
fileDir = pwd; % or your specific path / folder

S = dir(fullfile(fileDir,'Data*.txt')); % get list of data files in directory  

S = natsortfiles(S); % sort file names into natural order (what matlab dir does not well) , see FEX : 
           %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)
 
Nfiles = length(S);  % Number of files
combined_data= [];

%% Loop inside folder
for k = 1:length(S) % read data 
    fileName = S(k).name;
    data = readmatrix(fullfile(fileDir, fileName)); % or use a structure (S(k).data ) to store the full data structure
    
        % For the first file, store wavelengths in the first column
        if k == 1
            % store first time : Wavelengths + Intensities and add "mean" data to the right 
            combined_data = [data mean(data(:, 2:end),2)]; 
        else
             % next files
            combined_data= [combined_data data(:, 2:end) mean(data(:, 2:end),2)];  % then add Intensities + mean Intensity only (skipp first column Wavelengths) for the next files

        end
        
end

Answer 1

[Umeshraja]

Hi @Nadia

I understand you're looking to combine your measurements into a single matrix. Assuming your input files are text files with two columns (wavelength and intensity), here's a script to help you achieve this:

It combines by reading each sample file, Store wavelengths (only from the first file since they're constant), Store intensities from each file in subsequent columns

num_samples = 156;  % Number of files
num_rows = 5826;    

% Initialize with zeros
% First column will be wavelengths, other columns will be intensities
combined_data = zeros(num_rows, num_samples);

% Loop through each sample file
for i = 1:num_samples
        
        filename = sprintf('sample_%d.txt', i);

        % Assuming space-delimited text files with two columns:
        data = readtable(filename, "Delimiter", " ");
        
        % For the first file, store wavelengths in the first column
        if i == 1
            combined_data(:, 1) = data{:, 1};  % Wavelengths
        end
        
        combined_data(:, i + 1) = data{:, 2};  % Intensities

end

% Save the combined data 
save('combined_measurements.mat', 'combined_data'); 

For more details on using the 'readtable' function, you can refer to the following MATLAB documentation

https://www.mathworks.com/help/matlab/ref/readtable.html

Hope it helps!

[Nadia]

I am trying to work with it, but yet not sucessful. I also want to combine the mean of matricx. As 1 or 2 mean spectra per sample.

For example put together one matrix each for high and low intensties