My exported EDF file is up to 100 times bigger than my imported EDF file

Hi everyone,

I would like to import a set of EDF files, remove some artifacts and export again as EDF files (artifact free signal). The code in general works fine, but while the size of imported EDF files is 10-40 MB, the size of exported one is 600-800 MB. In the following code I excluded the lines related to artifact removal:

%% Preparation
clc
clear all
close all

SR=1000;                                  % Sampling Rate in Hertz
start_path = fullfile('EEG Analysis Code');
topLevelFolder = uigetdir(start_path);
allSubFolders = genpath(topLevelFolder);
remain = allSubFolders;
listOfFolderNames = {};

while true
    [singleSubFolder, remain] = strtok(remain, ';');
    if isempty(singleSubFolder)
        break;
    end
    listOfFolderNames = [listOfFolderNames singleSubFolder];
end

numberOfFolders = length(listOfFolderNames);
set(0,'DefaultFigureVisible','on')             %Change 'off' to 'on' if you want plotted figures to show, also refer to the last line of code (313)


% This block of code is where data collecting and processing begins ---------------------------------------
for k = 1 : numberOfFolders
    thisFolder = listOfFolderNames{k};                                      % This specifies the current folder for searching
    fprintf('Processing folder %s', thisFolder);
    filePattern = sprintf('%s/*.edf', thisFolder);                          % This obtains the names of EDF files in the current folder
    baseFileNames = dir(filePattern);                                       % This saves the name of EDF files in the current folder
    numberOfEDFFiles = length(baseFileNames);                               % This counts the number of EDF files present

    Summary_Names = {zeros(numberOfEDFFiles,1)};                            % Saves the mouse number for export to excel
    Summary_Data = zeros(numberOfEDFFiles,3);                               % Saves the mouse spikes/seizures for export to excel

    if numberOfEDFFiles >= 1                                                %This loop scans through all the EDF files and extracts data
        for f = 1:numberOfEDFFiles
            fullFileName = fullfile(thisFolder, baseFileNames(f).name);
            fprintf('\nProcessing EDF file %s \n \n', fullFileName);
            Mouse_Number = erase(baseFileNames(f).name, '.edf');            % This saves the mouse number (e.g. S20, S21, etc.)
            Data = edfread(fullFileName,'TimeOutputType','datetime');  % This is where EDF data is brought into matlab
            Data.Properties.DimensionNames = ["DateTime" "Variables"];  	% Converts EDF data to dd/mm/yyyy - hh/mm/ss format
            info = edfinfo(fullFileName);                         	% Saves parameters for the EDF file
            FilenameStr=info.Filename;
            PatientStr = info.Patient;
            RecordingStr = info.Recording;
            StartDateStr = info.StartDate;
            StartTimeStr = info.StartTime;
            Record = 1;                                                  	% EEG Record
            Lead = 1;                                                       % EEG Lead/Electrode
            Data.Properties.VariableNames(1)=cellstr('EEG');                    % in the case of wrong labeling of the signal
            Voltage_V = vertcat(Data.EEG{:});
            Time_total = info.NumDataRecords*info.DataRecordDuration;    	% Total of data points in the EEG trace
            Time_seg = Time_total/length(Voltage_V);                    	% Converts frequency to time in seconds
           

            % Date and Time Conversion --------------------------------------------------------------------
            Start_Date = Data.DateTime(1,1);                                % Start date and time for EEG trace
            End_Date = Data.DateTime(end,1);                                % End date and time for EEG trace
            DateTime_Vector = linspace(Start_Date,End_Date,length(Voltage_V)); % Creates array of date/time variables
            Timeframe = round(hours(End_Date-Start_Date),1);                % Rounds length of EEG trace to nearest 0.1 hours
            Timeframe_String = num2str(Timeframe);                          % Converts length of EEG trace to a string variable

        end
    end
end
%% edfheader
hdr = edfheader("EDF+");
hdr.Patient = PatientStr;
hdr.Recording = RecordingStr;
hdr.StartDate = StartDateStr;
hdr.StartTime = StartTimeStr;
hdr.Reserved = "EDF+C";
hdr.NumSignals = 1;
hdr.NumDataRecords = length(Voltage_V)/SR;
hdr.SignalLabels="EEG";
hdr.TransducerTypes="";
hdr.PhysicalDimensions="mV";
hdr.PhysicalMin = min(Voltage_V);
hdr.PhysicalMax = max(Voltage_V);
hdr.DigitalMin = [-32256];
hdr.DigitalMax = [32256];
hdr.DataRecordDuration = [seconds(1)];
%%
Fname=strrep(FilenameStr, '.edf', '_revised.edf');
edfwrite(Fname,hdr,Voltage_V,'InputSampleType',"physical");

What could be the reason for this issue?

Thanks in advance for your suggestions.

All the best,

Sadegh

Update 10/11/2023:

P.S. Below, I am sharing an EDF file that I am importing with edfread. After the export with edfwrite, it is more than 600 MB:

https://drive.google.com/file/d/1FD8MmqYhyvVuEi_Wjj-9cnf1d9usJlUW/view?usp=sharing

Answer 1

[Alex Wu]

Hi Sadegh,

This appears to be a deficiency in the software. When 'Reserved' in header is set to something other than an empty string "", edfwrite function will add an annotation as a placeholder for each data record, which is unnecessary. But even blank annotation placeholders are huge, especially in this case where you have 18000 data records which means 18000 redundant placeholders.

This issue will be addressed in an upcoming release. Thanks for reporting it.

As for the current workaround, if you want to reduce the saved file size, you can set hdr.Reserved to "" instead of "EDF+C".

[Sadegh Rahimi]

Hi Alex,

Thanks for your informative reply. I will set hdr.Reserved to "", hoping to see a significant reduction in the size of exported files.

Answer 2

[Sulaymon Eshkabilov]

Without seeing your data files and just quick looking at your code, this can be concluded. You are augmenting data from numberOfEDFFiles of files that means you are exporting the augmented data into a single file that answers your question of much larger exported file size.

[Sadegh Rahimi]

Thanks for your comment, but it is not the case. At the moment, I only import one EDF file (I put one EDF file in size of 10-40 MB there) and when I export it into EDF, it is always 600-800 MB.

Answer 3

[Peter Perkins]

I don't know much about EDF, and there's no fil attached, but my guess is that the culprit is

Voltage_V = vertcat(Data.EEG{:});

Post the file, or give details what's in it.

[Sadegh Rahimi]

Thanks for your comment. Below, I am sharing an EDF file that I am importing with edfread. After the export with edfwrite, it is more than 600 MB:

https://drive.google.com/file/d/1FD8MmqYhyvVuEi_Wjj-9cnf1d9usJlUW/view?usp=sharing