How to import large textfile

Hi EVERYONE I'm new to matlab, I've been trying to import large text file .asc to matlab. is there anyone can help. the file contains 59615 (24*24)matrix. I've attached small part of the file. When I import the whole file I get an error while if I import small part it works fine!! I don't think it's memory problem, can anyone help please???

[Sara]

There is no attachment and you need to cut and paste the error you get here, so people can see exactly what error you get

[dpb]

Didn't get the file attached.

How about letting us know what the error is? A 24x24 array is not a large file. More than likely there's something in the file causing a conversion error.

[Cedric]

It is not too large, but not small either. Assuming 10 chars per element (including separators, etc), it is more than 300MB

 >> 59615*24*24 * 10 / 1e6
 ans =
  343.3824

Could you provide a ~5MB sample?

[Rajab Omar]

here's the error I got: "import operation failed. the most likely reason is that there are unimortable cells in the selection. try to adding arule in the toolstrip to convert unimortable cells into numbers."

 I've a;ready excluded the raws with unimortable cells.

Answer 1

[Cedric]

Here is one way to proceed

 fId = fopen( 'omar_1.txt', 'r' ) ;
 buf = fscanf( fId, '%f', Inf ) ;
 fclose( fId ) ;

 bSize   = 24*24 + 1 ;
 nBlocks = length( buf ) / bSize ;
 data    = cell( nBlocks, 1 ) ;

 for bId = 1 : nBlocks
    bBnds     = ((bId-1)*bSize+2) : (bId*bSize) ;
    data{bId} = reshape( buf(bBnds), 24, 24 ).' ;
 end

At the end, you have your matrices in the cell array data:

 >> data
 data = 
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]

so data{1} is the first matrix. There are other approaches, but they a probably less efficient if your file is really large.

[Rajab Omar]

Hi Cedric Wannaz

Thanks a lot for your answer, it works fine. I've one more question if possible. I need to multiply the output for your code to (24*24matrix). When I convert the cell array to matrix, I got 1 large matrix (1430784*24). so what I'm after is how to use for loop to multiply the (24*24 matrix) to the (1430784*24) matrix, so it can take 24 rows at the time.

Best Regards

Rajab Omar

[Cedric]

Hi Rajab Omar,

You should keep the cell array. It is a cell array of matrices and not a cell array of numbers. The content of cell 1 is the first 24 x 24 matrix (or numeric array). To access it, use curly brackets (which mean "content of cell" with cell arrays). If you wanted to multiply the 5th matrix by a 24 x 1 (column) vector v, you would do it as follows

 result = data{5} * v ;

Just as a refresher, parentheses are for block indexing. When you block index a numeric array, you get another numeric array. When you block index a cell array, you get another cell array (or cell if it is just one element). So

 data(1)

is cell 1 of cell array data. It is a cell array as well, with one element. On the contrary,

 data{1}

is the content of cell 1 of the cell array data, which is a matrix (numeric array) in your context.

[Rajab Omar]

Thanks a lot Cedric Wannaz I've different question now, I'm not sure if it's OK to ask it within this question. I've got time series data and need to get the power spectral density (PSD), and the probability density function (PDF). I wrote code for PSD, then I realised that there's a function called (pwelch). However the results were not identical. I shall be grateful if you can suggest the best option to estimate both PSD and PDF for time series data. HERE'S WHAT I'VE DONE:

    %clf;clc;clear;
    VFF_1=VFF.*hanning(length(VFF));%%windowing
    N= length(VFF_1);
    dt=0.001;
    fs=1/dt
    tmax=(N-1)*dt;
    t=0:dt:tmax;

    tn=N/fs%%Total sample length
    fmin=1/tn
    fmax=0.5*fs;%%nyquist
    f=(1:N/2)/(N/2)*fmax;%define frequency
    f=f';
    length(f)
    y=fft(VFF_1);
    y(1)=[];
    py=abs(y(1:N/2)).^2;%%Define power spectrum
    length(py)
    figure
    plot(f,py)%%plot power spectrum against Frequency
    xlabel('Frequency (HZ)');title('Power Spectrum');ylabel('power');

REGARDS

[Cedric]

Hi, as it is a different topic, you should post a new question. You'll also get answers from people who are more proficient than I on this particular matter!

Regards,

Cedric