Clear Filters
Clear Filters

How to solve out of memory error when constructing sparse matrix

13 views (last 30 days)
I am construcing a stiffness matrix in finite element computation.
I have got i, j, and v vectors as input arguments for sparse.
Both i and j are 222263280-by-1 int32 vectors while v is a 222263280-by-1 double vector. The stiffness matrix is 1202454-by-1202454.
However I encountered out of memory error when I run the command S = sparse(i,j,v,m,n).
Are there any solutions or suggestions to address this problem?
Why does not sparse support tall array? I think this problem can be solved if tall array used.
Thanks.

Accepted Answer

Bruno Luong
Bruno Luong on 24 Sep 2023
Edited: Bruno Luong on 24 Sep 2023
You might try the syntax
nz = ... ; < 222263280
S = sparse(i,j,v,m,n,nz);
with nz is the estimation of non-zero elements of your matrix. If you build P1 FEM, it's the number of edges of your setup. You might use Euler formula to estimte it : https://www.mathsisfun.com/geometry/eulers-formula.html
or using
nz = size(unique([i(:) j(:)], 'row'),1)
Note that the byte size required to store a sparse matrix S is
if isa(S,'double')
elbyte = 8;
else
elbyte = 1;
end
Sbytesize = 8*(size(S,2)+1) + ... % Jc array
8*nnzmax(S) + ... % Ir array
elbyte*nnzmax(S) % data value array
So if you deal well, your S will take less than 3.566 Gb (I assumz nz < 222263280),
>> (1202454+1)*8+222263280*8+222263280*8
ans =
3.5658e+09
plus the space of I, J, V. It will require 7 Gb to build it. If your have other suff around 16 Gb become a little tight.

More Answers (2)

Matt J
Matt J on 24 Sep 2023
It's a 5 GB matrix. Are you sure you have enough RAM?
  4 Comments
lala
lala on 24 Sep 2023
No, I need help. 😭
since all i, j, and v can be maintained in RAM, why sparse failed? Does sparse need more memory than i, j, and v?
Torsten
Torsten on 24 Sep 2023
If I clear all variables except i, j, and v for sparse, it can construct the sparse matrix successfully.
This should show you that it's not possible with the RAM available.

Sign in to comment.


John D'Errico
John D'Errico on 24 Sep 2023
Edited: John D'Errico on 24 Sep 2023
Remember that i and j and v also take memory.
(222263280*4*2 + 222263280*8)/1024^3
ans = 3.3120
So those three variables alone require 3.3 GB. And then the sparse array would require 5 GB. Next, you need to factor in that MATLAB will require CONTIGUOUS memory for any arrays. And MATLAB itself, and your OS take up memory. So I can understand that you may just have room to create that array, if you clear everything else. But you still won't be able to do virtually anything with it once built.
This is not really a problem of needing support for tall arrays to build a sparse matrix. That is just a crutch that will encourage you to build larger arrays yet. If you are going to work with these problems on that computer, then you desperately need to get more RAM. RAM is cheap these days. Honestly, I would suggest 16 GB of RAM is just barely a viable amount these days if you are doing any serious work in MATLAB.
John's law: Computer problems always expand to just a little larger than the memory available on the largest computer you can find.
(So, yes, according to the law I just gave you, even if you do get more RAM, you will quickly decide to solve larger problems yet. Since who wants to solve small problems? But it gives you a few years of grace.)

Categories

Find more on Loops and Conditional Statements in Help Center and File Exchange

Products


Release

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!