TXT Data Extraction Help

Hello, I have some questions involving data abstraction from several text files.

Attached I have 3 text-files, each with two columns. In both the Loading.txt and Unloading.txt files are force and displacement data from AFM measurements. I want to extract the data from both and put them in a single 2 column matrix of force and displacement.

Then, in the Ft.txt I have time and force data. The force data have the exact same values of force as from the force displacement data. I want to extract this data, and line it up with the force displacement data, so I can get a two column time, displacement matrix. Not really sure how to go about this, since I am relatively new to Matlab.

Thanks

[dpb]

Well, putting the two together isn't too tough; the problem I see is that there are multiple readings that are duplicated for the force; how was that file created vis a vis the others? It would seem there should be a known relationship a priori?

Answer 1

[dpb]

OK, had a little time to explore the data enough to figure out what appears to be going on...a simplistic way since there turns out to be only a single minimum force value--

fvt=textread('fvt.txt','','headerlines',1);   % read the files; I used _much_ shorter names :)
lod=textread('loading.txt','','headerlines',1);
unlod=textread('unloading.txt','','headerlines',1);
lu=[lod(:,2);unlod(:,2)];    % concatenate the loading/unloading files (presume that order?)

At this point, I then plotted various views to get an idea what had, the most significant of which turned out to be (something you already knew :) ):

plot([fvt(:,2) lu]);

which did show the order in the two is correct and that the two force vectors do have same shape, just offset. So, given that there's a single minimum point, the easiest way to arrange them to line up on top of each other is:

[~,i1]=min(lu);          % find location of minimum in the concatenated loading vector
[~,i2]=min(fvt(:,2));    % and in the time/force vector
lu=circshift(lu,i2-i1);  % and make them line up with each other...
find(lu==min(lu))==find(fvt(:,2)==min(fvt(:,2)))  % just make sure they do agree now...
tdisp=[fvt(:,1) lu];      % ok make a new time/displacement array from the shifted and the time
figure, plot(tdisp(:,1) tdisp(:,2))  % show what it looks like...

NB: I used the deprecated textread because it a) doesn't need the extra fopen/fclose step of textscan and returns an "ordinary" double array instead of cell array which is convenient to eliminate a couple of steps. You can use importdata or dlmread or some of the other routines for higher-level simple files as desired. I just remember textread well-enough to not think of the others much being am such an old fogey... :)

If it weren't exact match I'd follow something similar to the above with perhaps average locations or whatever. I looked at simple cross-correlation to estimate lag; over the full vector it gets smushed out so much that estimated shift was only about half that needed. If one were to take a much shorter section around the peaks alone it would likely improve the estimate significantly but since it is deterministic the fixed expression used works first time. The alternate would be useful if there were some measurement noise or similar so the two didn't precisely match identically.

[mattyice]

Hey, thanks for the response. Before I try what you suggested, I just want to show you what I had done before seeing your response:

load='load.txt';
unload='unload.txt';
fvt='Fvt.text';
M =dlmread(load,'\t',1,0);
N =dlmread(unload,'\t',1,0);
O =dlmread(fvt,'\t',1,0);
A =[M;N]; 'combine load/unload in single 2column matrix
B =sortrows(A,2); 'sort load/unload data with respect to column 2 (force data)
C =sortrows(O,2); 'sort Fvt data with respect to column 2 (force data)
D = [B C]; 'horizontal concatenation of B & C
E =sortrows(D,3); 'Sort matrix D with respect to time
x=E(:,3); 'place time values into x (column 3)
y=E(:,1); 'Place displacement values into y (column 1)
plot(x,y);

Attached is the plot I get. As you can see, the sine function is mostly there, but it seems there were some repeated force values that, and so the were plotted in the wrong place. How would I go about fixing this, so the right displacement values are paired with the right time values.

Thank you

[dpb]

Don't think you want to do any sorting at all...the data are in sequential order, aren't they? That's what I presumed and it looks pretty reasonable to me...

[mattyice]

Except there is an offset of the force data on the load/unloading and the Force vs. time data. Ideally, if I could find a way to get the code to match the first Force value in the load/unloading matrix (A, column 2, row 1) with the corresponding force in the FvT matrix (O, column 2, row 218), move this row and all rows below row 218 to the top of the matrix and concatenate the ones above it, to the bottom, I would get a more smoother curve. If you look at the left side of my graph, the plot is much more smooth than yours, so this adjustment must be accounted for

[dpb]

That's what mine does...excepting I used the minimums at 95 and 56 to do the alignment. The problem is when you sort by increasing force you're going to reverse time observations as you've observed. I don't know the 'spearmint but that just doesn't seem at all correct to me. Once you do that sorting, you don't have any place to put those that are out of order back in line again without making up a sequence that seems to me to not be so...

Here's the two -- as said, what I've done is align the two minimum points but that's the same as the average of the max points to the right. circshift does the rotation of the data from that position back around and joins it to the other end just as you describe wanting to do.

When you sort, however, you mix up the order, not by time but by force. Unless you're traveling with Dr Who, I don't know how to make that actually occur.

[mattyice]

Oh. I got it. Thank you so much!!

[dpb]

So, if this solved the problem, go ahead and "Accept" the answer to close the thread if nothing else.