How to solve out of memory error when constructing sparse matrix
6 views (last 30 days)
I am construcing a stiffness matrix in finite element computation.
I have got i, j, and v vectors as input arguments for sparse.
Both i and j are 222263280-by-1 int32 vectors while v is a 222263280-by-1 double vector. The stiffness matrix is 1202454-by-1202454.
However I encountered out of memory error when I run the command S = sparse(i,j,v,m,n).
Are there any solutions or suggestions to address this problem?
Why does not sparse support tall array? I think this problem can be solved if tall array used.
Bruno Luong on 24 Sep 2023
Edited: Bruno Luong on 24 Sep 2023
You might try the syntax
nz = ... ; < 222263280
S = sparse(i,j,v,m,n,nz);
with nz is the estimation of non-zero elements of your matrix. If you build P1 FEM, it's the number of edges of your setup. You might use Euler formula to estimte it : https://www.mathsisfun.com/geometry/eulers-formula.html
nz = size(unique([i(:) j(:)], 'row'),1)
Note that the byte size required to store a sparse matrix S is
elbyte = 8;
elbyte = 1;
Sbytesize = 8*(size(S,2)+1) + ... % Jc array
8*nnzmax(S) + ... % Ir array
elbyte*nnzmax(S) % data value array
So if you deal well, your S will take less than 3.566 Gb (I assumz nz < 222263280),
plus the space of I, J, V. It will require 7 Gb to build it. If your have other suff around 16 Gb become a little tight.
More Answers (2)
John D'Errico on 24 Sep 2023
Edited: John D'Errico on 24 Sep 2023
Remember that i and j and v also take memory.
(222263280*4*2 + 222263280*8)/1024^3
So those three variables alone require 3.3 GB. And then the sparse array would require 5 GB. Next, you need to factor in that MATLAB will require CONTIGUOUS memory for any arrays. And MATLAB itself, and your OS take up memory. So I can understand that you may just have room to create that array, if you clear everything else. But you still won't be able to do virtually anything with it once built.
This is not really a problem of needing support for tall arrays to build a sparse matrix. That is just a crutch that will encourage you to build larger arrays yet. If you are going to work with these problems on that computer, then you desperately need to get more RAM. RAM is cheap these days. Honestly, I would suggest 16 GB of RAM is just barely a viable amount these days if you are doing any serious work in MATLAB.
John's law: Computer problems always expand to just a little larger than the memory available on the largest computer you can find.
(So, yes, according to the law I just gave you, even if you do get more RAM, you will quickly decide to solve larger problems yet. Since who wants to solve small problems? But it gives you a few years of grace.)