Export matched lines from two text files
2 views (last 30 days)
Show older comments
I need to identify the same lines between the two text files, mwithrm21.txt and virgomrmdist.txt, based on column 7 of each files. These matches should then be exported into a new text file, while removing the matched lines from mwithrm21.txt.
I have attached the text files.
I drafted the code below:
content1 = fileread( 'mwithrm21.txt' ) ;
content2_rows = strsplit( fileread( 'virgomrmdist.txt' ), sprintf( '\n' )) ;
found = cellfun( @(s)~isempty(strfind(content1, s)), content2_rows ) ;
output_rows = content2_rows(found) ;
fId = fopen( 'similarvclf.txt', 'w' ) ;
fprintf( fId, '%s\n', output_rows{:} ) ;
fclose( fId ) ;
output_rows = content2_rows(~found) ;
fId = fopen( 'mwithrm21_new.txt', 'w' ) ; % Remove the '_new' for overwriting original.
fprintf( fId, '%s\n', output_rows{:} ) ;
fclose( fId ) ;
But, I do not know how to make it specific to only searching column 7 and then exporting the entire matched line to a new text file.
6 Comments
Cedric
on 15 Aug 2015
Edited: Cedric
on 15 Aug 2015
Actually, you say that you want to match rows based on column 7 only, but when they match the other columns don't always match. What do you want to have in the output?
For example:
file1 : 188.83785|27.56214|-14.4|18.931|0.398|~|SDSSJ123521.05+273343.6
file2 : 188.83785|27.56214|18.931|0.398|-14.4|~|SDSSJ123521.05+273343.6
Should we export both?
Accepted Answer
Cedric
on 15 Aug 2015
Edited: Cedric
on 16 Aug 2015
Here is a first draft. Test it and let me know if anything is unclear or doesn't work.
% - Read files content as strings.
content1 = fileread( 'mwithrm21.txt' ) ;
content2 = fileread( 'virgomrmdist.txt' ) ;
% - Extract last column of each content.
codes1 = regexp( content1, '[^|]+(?=(\s|$))', 'match' ) ;
codes2 = regexp( content2, '[^|]+(?=(\s|$))', 'match' ) ;
% - Matches codes.
[isMatch_1in2, match_posIn2] = ismember( codes1, codes2 ) ;
% - Split content. Careful, whatever generates these files still uses
% carriage returns (\r) only.
rows1 = strsplit( content1, char(13) ) ;
rows2 = strsplit( content2, char(13) ) ;
% - Output matches (version mwithrm21.txt). Use new line chars (\n) as
% joint instead of carriage returns, change if you prefer \r.
fId = fopen( 'matches.txt', 'w' ) ;
fwrite( fId, strjoin( rows1(isMatch_1in2), '\n' )) ;
fclose( fId ) ;
% - Output non-matching rows of file 1.
fId = fopen( 'mwithrm21_reduced.txt', 'w' ) ;
fwrite( fId, strjoin( rows1(~isMatch_1in2), '\n' )) ;
fclose( fId ) ;
% - Output non-matching rows of file 2. Eliminate matching rows first.
rows2(nonzeros( match_posIn2 )) = [] ;
fId = fopen( 'virgomrmdist_reduced.txt', 'w' ) ;
fwrite( fId, strjoin( rows2, '\n' )) ;
fclose( fId ) ;
EDITs :
- Replaced match_posIn2(match_posIn2~=0) with nonzeros( match_posIn2 ) after reading an answer by Matt J in another thread that mentions NONZEROS.
7 Comments
Cedric
on 17 Aug 2015
Edited: Cedric
on 17 Aug 2015
Just replace the patterns in the calls to REGEXP with:
'(?<=(\s|^))[^|]+'
This means: match
- One or more character (as many as possible) different from '|'.
- Preceded by either a white space (which, in your case, is the carriage return of the previous line) or the beginning of the string.
PS1: if execution speed is important, you should profile Per's solution and mine, to see which one is the fastest in your specific case.
PS2: the previous patterns
'[^|]+(?=(\s|$))'
meant: match
- One or more character (as many as possible) different from '|'.
- Followed by either a white space (which, in your case, is the carriage return after column 7) or by the end of the string.
PS3:
- [abc] is a set of characters to match (not a string, it means 'a' or 'b' or 'c').
- [^abc] is a set of characters not to match.
- + means "one or more (as many as possible) what precedes".
- (?<=...) is a positive look-behind for whatever is between = and ).
- (?=...) is a positive look-forward for whatever is between = and ).
- (..|..) is an OR operator.
- \s means "white space" which includes spaces, tabs, carriage returns, new lines characters.
- ^ codes for the beginning of the string.
- $ codes for the end of the string.
With that, you can decipher the patterns in principle.
More Answers (2)
per isakson
on 16 Aug 2015
Edited: per isakson
on 16 Aug 2015
Here is an example of a different approach to solve the task. The two output files, mwithrm21_reduced.txt and matches.txt, are identical besides the new line characters.
function et = cssm()
% et(1) = cssm_1();
et(2) = cssm_2();
end
function et = cssm_2()
tic
fid = fopen( 'mwithrm21.txt', 'rt' );
rows1 = textscan( fid, '%s', 'Delimiter','\n' );
fseek( fid, 0, 'bof' );
codes1 = textscan( fid, '%*s%*s%*s%*s%*s%*s%s', 'Delimiter','|' );
fclose( fid );
%
fid = fopen( 'virgomrmdist.txt', 'rt' );
codes2 = textscan( fid, '%*s%*s%*s%*s%*s%*s%s', 'Delimiter','|' );
fclose( fid );
%
ism = ismember( codes1{1}, codes2{1} );
%
fid = fopen( 'matches.txt', 'wt' );
fprintf( fid, '%s\n', rows1{1}{ism} );
fclose( fid ) ;
%
fid = fopen( 'mwithrm21_reduced.txt', 'wt' );
fprintf( fid, '%s\n', rows1{1}{not(ism)} );
fclose( fid );
et = toc;
end
r r
on 11 May 2021
I have two files in which there are numbers in the first column that are similar and I want to print the line that matches and differs in the number of the first column in the two files:
%%%%%%%%%%%%%%%%%%%%%%% Fiel.1
fid1 = fopen( 'E1.txt', 'rt' );
T1 = textscan(fid1,'%s', 'delimiter', '\n');
%codes1 = textscan( fid1, '%*s%*s%*s%*s%*s%*s%s', 'Delimiter','|' );
fclose( fid1 );
%%%%%%%%%%%%%%%%%%%%%%%%%%Fiel.2
fid2 = fopen( 'G1.txt', 'rt' );
T2 = textscan(fid2,'%s', 'delimiter', '\n');
%codes2 = textscan( fid2, '%*s%*s%*s%*s%*s%*s%s', 'Delimiter','|' );
fclose( fid2 );
%%%%%%%%%%%%%%%%%%%%%%%%%%%
T1s = char(T1{:});
T2s = char(T2{:});
%Similar data between two files::
%[C,ix,ic] = intersect(T1s,T2s,'rows')
%Differences data between two files::
[B,ib,ib] = visdiff(T1s,T2s,'rows')
%%%%%%%%%%%%%%%%%%%%print output:::
fid = fopen( 'Similar.txt', 'wt' );%Print all similar lines
fprintf('%s\n',C)
fclose( fid ) ;
fid = fopen( 'Different.txt', 'wt' );%Print all different lines
fprintf('%s\n',B)
fclose( fid );
0 Comments
See Also
Categories
Find more on Low-Level File I/O in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!