How to have a code restart with a different file after finishing with a different file?

3 views (last 30 days)
Im still new to these more complex ideas in matlab, I have used it plenty for basic graphing of data but now im trying to do some more complex stuff. I was given a lot of data, like 40k txt files to process. Its all single beam bathymetry data, I want to process this data then export the clean data as a txt so I can use a different program to visualize the data and export as a geotiff to GIS. I have a solid code right now for smoothing and cleaning the data and exporting the data as a txt.
Ill attach the code but to explain a bit more, the code smooths the data then exports the new depth, lat and long to a csv file in a different folder. The original data file is also moved to a different folder so I know which ones I've done. What I want to do next is figure out how to do this with a bunch of files with one click. I could rename the file in the matlab code and run it every time but instead I want the code to search the folder for files with the same extension, run the code, create a txt, then move both files to different folders. When its done I want it to do it again with the next file. Hopefully my description makes sense. Im not sure where to start, im assuming I will need some kind of for loop but ive never done that before and could use some help getting started. Thanks!
%Variables from CSV
%File to be read
D = readmatrix("SingleBeam1.csv");
%Latitude
lat = D(:,1);
%Longitude
lon = D(:,2);
%Depth
dep = D(:,3);
%Altitude
alt = D(:,4);
%True depth of sea floor (Depth + altitude)
Dep = alt+dep;
%smoothed true depth
depS = smoothdata(Dep,'movmedian',500);
%%/////////////////////////////////////////////////////////////////////////
%%Plots regular depth and smoothed depth
% figure
% plot(Dep)
% hold on
%
% plot(depS)
% hold off
%%/////////////////////////////////////////////////////////////////////////
% %Plots the uncorrected bathymetry with normal plane
% figure
% plot3(lat, lon, depS, '.')
% hold on
% grid on
% view(35,60)
% title('Bathy Lines1')
%/////////////////////////////////////////////////////////////////////////
%compute the normal to the plane and a point that belongs to the plane
downsampling = 100; % to reduce risk of out of memory" in affine_fit
ind = (1:downsampling:numel(lat));
XYZ = [lat(ind) lon(ind) depS(ind)];
[n_1,~,p_1] = affine_fit(XYZ);
%plot the two adjusted planes
[Xf,Yf] = meshgrid(lat(ind),lon(ind));
%fitted plane
% Zf = - (n_1(1)/n_1(3)*Xf+n_1(2)/n_1(3)*Yf-dot(n_1,p_1)/n_1(3));
% surf(Xf,Yf,Zf,'edgecolor','none','facecolor','red','facealpha',0.25);
figure
depf = - (n_1(1)/n_1(3)*lat+n_1(2)/n_1(3)*lon-dot(n_1,p_1)/n_1(3));
plot3(lat, lon, depf, '.')
hold on
grid on
view(35,60)
title('Average Surface')
%%////////////////////////////////////////////////////////////////////////
%%Create and format final files
%Move single beam file to clean folder
movefile 'SingleBeam1.csv' 'Clean'
%Save the important variables as a 3x1 matrix
M = [lat(:), lon(:), depf(:)];
%export that matrix to a new txt file, now ready for NaviModel
writematrix(M, 'SingleBeam1Corrected.txt','Delimiter', 'tab')
type 'SingleBeam1Corrected.txt'
%move corrected file to corrected folder
movefile 'SingleBeam1Corrected.txt' 'Corrected CSV'
UPDATE: I found the below code online that was helpful in reading all of my files. I was able to read the files then continue with processing the data. However it combined all of the files into one output file, my new question is how to create a new txt file for each csv file that is read, and additionally how do I name them in sequential order? Thanks again!
file = dir('*.csv');
num_files = length(file);
sorted = sort({file.name});
for A = 1:num_files
table = table2array(readtable(sorted{A}));
T = table(:, 1:4);
lat = T(:,1);
%Longitude
lon = T(:,2);
%Depth
dep = T(:,3);
%Altitude
alt = T(:,4);
end

Accepted Answer

Stephen23
Stephen23 on 10 Mar 2024
Edited: Stephen23 on 10 Mar 2024
"However it combined all of the files into one output file, my new question is how to create a new txt file for each csv file that is read, and additionally how do I name them in sequential order?"
I am guessing that you expect the filenames to be in alphanumeric order. Note that if each file is processed independently of each other, the order in which they are processed is not important. However, if you want to sort filenames into alphanumeric order then you can download my FEX submission NATSORTFILES:
and use it something like this:
P = 'C:/somedir/subdir'; % absolute or relative path to where the CSV files are saved
Q = 'C:/somedir/subdir'; % absolute or relative path to where the TXT files will be saved
S = dir(fullfile(P,'*.csv'));
S = natsortfiles(S); % must DOWNLOAD
for k= 1:numel(S)
F = fullfile(S(k).folder,S(k).name);
M = readmatrix(F);
lat = M(:,1);
%Longitude
lon = M(:,2);
%Depth
dep = M(:,3);
%Altitude
alt = M(:,4);
... processing your data
X = ... your output matrix
[~,F,~] = fileparts(F);
F = sprintf('%s.txt',F);
writematrix(X,fullfile(Q,F))
end

More Answers (0)

Categories

Find more on Data Import and Analysis in Help Center and File Exchange

Products


Release

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!