Saving image files using fwrite creates huge files

Hi

I'm looking at using fwrite to save the (transformed) matrix components extracted from image files using fwrite but no amount of tweaking gets the resulting file to a manageable size, with size growing linearly (unsurprisingly) as the matrix dimension grows. For example, a 3000 x 4000 pixel PNG (18.4 Mb in size) saved as uint8 using fwrite (opening the file container as Wb) takes up 34.2 Mb in size, raw.

The sequence is the familiar one (XXX the file ID, YYY the data matrix to store)

fid = fopen(XXX,'Wb');

fwrite(fid,YYY,'uint8');

fclose(fid)

Using compression the resulting file shrinks but still requires over 20 Mb. The problem compounds massively if the file read is a MP4. Q is if there is any alternative method/routine to explore?

Thanks.

Answer 1

[Image Analyst]

A 12 million pixel image, if it's RGB, will take up 36 million bytes on disk assuming you use fwrite() to write just raw pixel data and no header/meta data. If you use imwrite() to compress it losslessly into a PNG format, it will be less, and 18 MB sounds reasonable. JPG will compress it even more but you'll change the pixel values and it won't look as good when recalled from disk.

If you want an uncompressed video with 12 megapixels and 15-30 frames per second, yes, it's likely the video file will be huge - in the GB or TB range depending on the length of the movie.

You have to ask yourself what size zcreen will people be playing my movie on. If it's a full size HDTV screen at 1920x1080, then there is no sense saving out the full resolution. Your uses will never see the extra resolution. Simply resize the image before adding the frame to the movie.

[Image Analyst]

For what it's worth, a few demos are attached.

[Eduardo Salazar]

Thanks, I'll have a look. FWIW I've been using a combination of Lanczos 2 and decimation (that is, downsampling) and Lanczos 3 + upsampling (or interpolation, to restore the frames taken out) using a predictive algorithm that rebuilds any missing frame (say, q) by looking at the [q-1] and [q+1] frames. The algorithm can be perfected, e.g. by sub-sampling areas of the decimated frames and use a distance metric to better calibrate the interpolation during the upsampling phase. But that would require saving the sub-sampled pixels, which would add to the file size (precisely what I'm trying to avoid).

FWIW on my comment above, Laczos 2 (down) and Lanczos 3 (up) works better (yields less visible artifacts) than Lanczos 2 (down-up) or Lanczos 3 (down-up).

[Image Analyst]

Again, do you need the full 3000x4000 resolution for your movies? Tell us what kind of monitor people are going to view your movie on. Do they have some kind of 12 megapixel display?

[Eduardo Salazar]

There is the possibility of 5K rendering so a high resolution might be called for, although agree it's not essential.

Answer 2

[Walter Roberson]

is if there is any alternative method/routine to explore?

No, there is not.

Every method that encodes data into less data, in a more-or-less reversible manner, is a compression algorithm.

3000*4000*3 / 1024 / 1024

ans = 34.3323

Any method that represents those 34 megabytes as less data, is a compression algorithm.

You have ruled out using compression algorithms, so you have ruled out representing your 34 megabytes as a smaller size.

[Walter Roberson]

Using compression the resulting file shrinks but still requires over 20 Mb.

You should mentally reframe that.

When you used the compression method that you used the compression gave that result.

However, there are an infinite number of compression methods. Every one-to-one function that maps at least one value to a value that takes less space to represent, is a compression method.

So the particular one you used gave the 20 Mb result, but that does not mean there are not better compression methods available. For example you could use the compression method that PNG does; https://en.wikipedia.org/wiki/Portable_Network_Graphics#Compression

[Eduardo Salazar]

Correct. It's the compression method that gave that result (in this case, gzip). And of course there are N number of compression methods, each delivering a particular outcome. I know that.

I'm experimenting with several methods. So far for single images I get a result within range of the original file, so that's fine. The issue arises with movies, so multiple images or frames.