problem in parsing data

5 views (last 30 days)
Ricky
Ricky on 7 Jul 2013
Hello Everyone,
I am reading a binary file using MATLAB whose size is around 96 MB. After I read in the binary file I convert it into a MATRIX of 11563 rows and 8192 columns. I have written a code for data parsing which takes the first row and the first 12 columns are taken as header of data and the rest is taken as the actual data. Problem is while I am parsing the the initial 1000 rows are parsed quickly and then MATLAB slows down considerably. How can I increase the speed of my MATLAB code. I am storing the data in predefined cell array.
So for example after I read in Binary file my data is uint8 which I save into RawData variable uint8. My MATLAB has been running for 3 hours and I have only been able to extract some 5000 rows using my code.
  3 Comments
Ricky
Ricky on 8 Jul 2013
The reason for preallocating a cell array is the size of array inside each cell is not fixed. In my cell array I have two structures , the first structure contains the header of my data and second structure contains the actual data. Depending upon the sensor used my second structure can be of < 1*8192 uint8 > or < 1*16384 uint8 >. The sensor type is selected using a GUI so I cannot preallocate the array size.
You are right I am looping through my input MATRIX and storing each row in a cell array.
per isakson
per isakson on 8 Jul 2013
Edited: per isakson on 8 Jul 2013
Did you watch the Windows Task Manager | Performance while your code is running?
It's the fields of the actual and the header data that needs to be pre-allocated.
How much RAM does the computer have?

Sign in to comment.

Answers (2)

Image Analyst
Image Analyst on 7 Jul 2013
Why are you using a cell array? That probably slows it down. Also, if you're displaying the data with image() or imshow() without clearing out prior images you've stored, then that will slow it down also. Use "cla" to clear the old images/matrices out of the axes before you display a new one.
  2 Comments
Ricky
Ricky on 8 Jul 2013
Actually this is not an image data but ultrasonic sensor data.
Image Analyst
Image Analyst on 8 Jul 2013
I still don't see why you can't just use a regular structure for your header data, and then a uint8 or uint16 array for your data. I do that all the time. I just read stuff one at a time and load up a header structure, then when it's time to get the data, just use fread() to load all the data into one 2D array all in one fell swoop (no looping required).

Sign in to comment.


Muthu Annamalai
Muthu Annamalai on 8 Jul 2013
Edited: Muthu Annamalai on 8 Jul 2013
You want to cast your data to right type. Default numeric type in MATLAB in double. You may want to use uint8() on your fread() calls so that you only store byte-sized chunks into your variable.
If you do this consistently you should get a 8x reduction in the data size.
Also as others have commented, it is always good to share your code to provide context to your problem!

Categories

Find more on Structures in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!