Saving many variables as NetCDF makes file too large. How to reduce?
22 views (last 30 days)
Show older comments
Hello all,
I would REALLY appreciate any help on this, as I looked online but didn't find straight answers. I am trying to save many variables of different sizes as NetCDF format in this way:
numrow = 576;
numcol = 48;
numlayers = 125;
numtime_save = numtime + 1; %This is usually a value of '8'
%%Define the dimensions
netcdf.setDefaultFormat('NC_FORMAT_CLASSIC') ;
ncid = netcdf.create(files_name,'NC_WRITE');
dimidrow = netcdf.defDim(ncid,'rows',numrow);
dimidcol = netcdf.defDim(ncid,'length',numcol);
dimidlay = netcdf.defDim(ncid,'layers',numlayers);
dimidtime = netcdf.defDim(ncid,'time',numtime_save);
%%Define the name of variables, with dimensions
%%Question: do these all really have to be double precision???
varmask = netcdf.defVar(ncid,'cloudmask','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_sigma = netcdf.defVar(ncid,'cloudmask_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear = netcdf.defVar(ncid,'cloudmask_clear','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear_sigma = netcdf.defVar(ncid,'cloudmask_clear_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar_sigma = netcdf.defVar(ncid,'cloudmask_radar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both = netcdf.defVar(ncid,'cloudmask_both','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both_sigma = netcdf.defVar(ncid,'cloudmask_both_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar = netcdf.defVar(ncid,'cloudmask_lidar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar_sigma = netcdf.defVar(ncid,'cloudmask_lidar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight = netcdf.defVar(ncid,'height','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight_sigma = netcdf.defVar(ncid,'height_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varN = netcdf.defVar(ncid,'N','NC_BYTE',[dimidrow dimidcol dimidtime]);
%%end the variable naming and dimensions, then actually input
%%variables into netcdf file, then close the netcdf file
netcdf.endDef(ncid);
netcdf.putVar(ncid,varmask,cloudmask);
netcdf.putVar(ncid,varmask_sigma,cloudmask_sigma);
netcdf.putVar(ncid,varmask_clear,cloudmask_clear);
netcdf.putVar(ncid,varmask_clear_sigma,cloudmask_clear_sigma);
netcdf.putVar(ncid,varmask_radar,cloudmask_radar);
netcdf.putVar(ncid,varmask_radar_sigma,cloudmask_radar_sigma);
netcdf.putVar(ncid,varmask_both,cloudmask_both);
netcdf.putVar(ncid,varmask_both_sigma,cloudmask_both_sigma);
netcdf.putVar(ncid,varmask_lidar,cloudmask_lidar);
netcdf.putVar(ncid,varmask_lidar_sigma,cloudmask_lidar_sigma);
netcdf.putVar(ncid,varheight,height);
netcdf.putVar(ncid,varheight_sigma,height_sigma);
netcdf.putVar(ncid,varN,N);
netcdf.close(ncid);
clear mex
However, when I save it this way, the file size is 1.33 GB! That is huge...If I save the files as:
save(filename, 'cloudmask','cloudmask_sigma','cloudmask_clear','cloudmask_clear_sigma',...
'cloudmask_radar','cloudmask_radar_sigma','cloudmask_both','cloudmask_both_sigma',...
'cloudmask_lidar','cloudmask_lidar_sigma','height','height_sigma','N');
this file has a size of 16 MB. I would REALLY like to save in NetCDF format, because my boss needs it this way. I have many files to save, this is just an example but it is inside of a loop. PLEASE HELP!
5 Comments
Walter Roberson
on 28 Mar 2018
You should post that as an Answer so we can vote for it. (You did the work of putting together the answer that would be of use to other people.)
Accepted Answer
More Answers (0)
See Also
Categories
Find more on NetCDF in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!