rmdir frequently fails with the 'MATLAB:RM​DIR:SomeDi​rectoriesN​otRemoved' error

I'm using matlab.unittest.fixtures for unit testing with matlab.unittest.TestCase. The operation involves file conversions, so I think it makes sense to use "setup" method for preparing a work folder and "teardown" for deleting the same folder.

The removal of the folder is done by rmdir function. It works fine, but 3 to 5 times per 10 trials, it fails to delete the folder with an error message 'MATLAB:RMDIR:SomeDirectoriesNotRemoved'.

This is essentially about the same issue, but putting rehash() after rmdir did not help. It's just unpredictable at the moment. I checked the folder's write permission with fileattrib function. Write permission is surely granted, but it still fails.

Only to illustrate the point, I wrote much simpler code. Interestingly, I successfully reproduced the problem.

clc;
str = 'TEMP_FOLDER';

for i = 1:100
    if isdir(str)
        rmdir(str,'s')
        rehash
    end

      mkdir(str);

      A = rand(10);
      save(fullfile(str,'A'),'A')

      [~,s] = fileattrib(str);
      fprintf('UserWrite %d\n',s.UserWrite);

      try
          rmdir(str,'s');
          rehash
      catch mex
          disp(mex)
          throw(mex)
      end
      fprintf('OK %d\n',i);

end

Although it's quite random, usually it fails within 10 trials.

UserWrite 1
OK 1
UserWrite 1
OK 2
UserWrite 1
  MException with properties:

      identifier: 'MATLAB:RMDIR:SomeDirectoriesNotRemoved'
         message: 'D:\xxxxx\TEMP_FOLDER could not be removed.'
           cause: {0x1 cell}
           stack: [0x1 struct]

D:\xxxxx\TEMP_FOLDER could not be removed.

Does anyone know how to solve this? I tested on Windows 7 (R2016a); the same code worked on OS X (R2016a).

[Walter Roberson]

As a data point: on R2016a on OS-X, none of the rmdir fail.

[Kouichi C. Nakamura]

Yes, I tried myself too, and the test code worked fine on R2016a on OS X.

Answer 1

[Philip Borghesani]

This is often caused by interaction with your virus scanner that happens to still be scanning / looking into files that may already have been deleted in the directory. Short of disabling your virus scanner the only solution is wait a short time and retry the operation. I suggest using the status outputs from rmdir to write a short loop something like this: (untested)

for try=1:4
  status=rmdir(str,s);
  if status==1
     break
  end
  sleep(.1*try) 
end
if status==0
  warning('mytest:leakedDir','Warning failed to clean up directory %s", str);
end

[Kouichi C. Nakamura]

Thanks! I examined the effect of pause duration as below. It's working, but looks like you need at least 1 sec or more. However, I'm not sure if this is caused by Antivirus software. When I checked the above code on Mac, no Antivirus software was running.

clc;
str = 'TEMP_FOLDER';

durations = [0 0.1 0.2 0.4 0.8];
c = zeros(10,5);

disp(datestr(now))
tic
for l = 1:10
    for k = 5:-1:1
        for i = 1:100
            if isdir(str)
                pause(2);
                if isdir(str)
                    rmdir(str,'s')
                    pause(2);
                end
            end

              if exist(str,'file')
                  delete(str)
              end

              mkdir(str);

              A = rand(10);
              save(fullfile(str,'A'),'A')

              try
                  status = rmdir(str,'s');

              catch exc

                  disp(exc.message)
                  status = -1;

                  pause(2)
                  rmdir(str,'s'); % try to delete it anyway after wait

              end

              switch status
                  case -1
                      disp(exc.message);
                      c(l,k) = c(l,k) + 1;
                  case 0
                      warning('mytest:leakedDir','Warning failed to clean up directory %s', str);
                      c(l,k) = c(l,k) + 1;
                  case 1
                      fprintf('OK %d\n',i);

              end

              pause(durations(k));

          end

          fprintf('Failed %d times for pause duration %f.\n',c(l,k),durations(k));
      end
  end

[Philip Borghesani]

Your test never fails on my Win7 machine running with Kaspersky AV and MATLAB R2015a or R2016a. This sort of thing is dependent on many software variables.

For any final work around I suggest retrying failed rmdirs instead of a fixed pause duration. It is generally impossible to determine with any certainty how long a pause would need to be and a temporary spike in system use could increase the needed duration. Your code will also run faster without a hard coded pause in each loop.

[Kouichi C. Nakamura]

Thanks a lot. Your report made me rethink of the culprit and I think I just found one! It's Dropbox. I put everything in Dropbox and it has been interfering file removal. When I paused Dropbox, I got no errors with the code at the top. But when I resumed Dropbox, I got loads of errors. And that was reproducible. Again Thank you for your input.

[Volodymyr Zenin]

Thank you, Philip Borghesani, for giving this solution - I had the same problem as the Kouichi C. Nakamura, and I suspect OneDrive in interference... By the way, please correct your answer: 1) try cannot be used as variable - it is kind of function in new Matlab; 2) there is no more such function as sleep, one should use pause instead.

Answer 2

[Andy Campbell]

Have you tried using the TemporaryFolderFixture or the WorkingFolderFixture?

[Kouichi C. Nakamura]

Wow, I did not know there are specific types of fixture already available. Good to know, thanks!

Documentation says

Before it deletes the folder, the fixture clears from memory the definitions of any MATLAB-files, P-files, and MEX-files that are defined in the temporary folder.

Yes, this sounds like related to my issue.