Motion tracking for research

Hi All,

First, thanks for any help in advance. I appreciate it. Alright, so I am looking for a way to plot the displacement of the larynx as someone is speaking over time. I have a high-speed camera where I am recording someone speak (their neck/larynx) at 300 FPS. I've imported the video into MATLAB, converted the frames to BW with nice thresholding so that I have just the edge of the neck including the larynx and nothing else, but am stuck at finding a way to identify the peak of the larynx and then tracking this displacement from frame to frame. Again, any help is greatly appreciated.

[Walter Roberson]

Could you post a few images? If the point if interest is not obvious, please indicate it in each case.

http://www.mathworks.com/matlabcentral/answers/7924-where-can-i-upload-images-and-files-for-use-on-matlab-answers

[Chris Garry]

Hi Walter,

Thanks for your comment. Here is a link to an example of what the first frame of the recording of the neck would look like after my pre-processing. Every other frame after the first will look the same except the larynx (big bump in the neck) will change vertical position since the subject is speaking. The chin (at the top of the picture) may move a bit and get into the frame, but nothing too drastic since we will be securing motion of the test subjects in future trials:

http://imageshack.us/photo/my-images/508/larynx.png/

[David Young]

The image posted looks like a synthetic image, not the real thing. It's very likely that techniques developed using it will fail on the real images from the video - a version of the "toy-worlds" problem familiar to AI researchers. I think it would be better to post an example (or several) from the actual data.

[Chris Garry]

Will do. I've been quite busy lately but will post when I have some time.

Answer 1

[Walter Roberson]

Row-wise, find() the location of the first black pixel. That list of pixels will have a minima in it, where minima is defined as the number on both sides being larger, which is a fairly simple logical test.

Let's see... if the white pixels are 1 and the black are 0, then you can find the location of the black area as cumsum() along rows.

[Chris Garry]

Hi Walter,

I am a bit confused. We are only interested in the bulge in the center of the image protruding from the neck. It will change vertical position in consecutive frames, like your "Adam's Apple" moves as you speak, since that's what it is. It seems as though what you're suggesting is an interest in the entire black portion of the image, as indicated when you said "you can find the location of the black area as cumsum() along rows," which is not the case. We want to identify the coordinates of the peak of the "Adam's Apple," something like a local maximum in calc terms, in every frame. That is all. I'm not sure how using find() to locate the first black pixel in the image or using cumsum() will help us identify where the larynx peak is located. Sorry for the confusion and thanks again for your help and patience.

[Walter Roberson]

If the white pixels have value 1 and the black pixels have value 0, then since for any one row the area stays black once it has turned black, then the sum() of the pixel values along the row would give you the same value as the index of the last white pixel. But you can sum(IMG,2) to do all the rows in a simple fast command, whereas you would have to loop if you used find().

I should have said sum() before, by the way, not cumsum().

Once you have the vector of "number of white pixels before the black is encountered" for each row, then except perhaps for the chin at the top, with the shape of the image you have, the peak of the larynx would correspond to the place where that count vector has a local minimum:

CV = sum(IMG,2);

lpeak = 1 + find(CV(2:end-1) > CV(1:end-2) & CV(2:end-1) > CV(3:end));

The "1 +" is there because the test for the minima does not start immediately at the left edge (because there is no "pixel before" to compare the first pixel to to determine that you are in a local minima)

The larynx peak would be at row lpeak (counting down from the top), and at column CV(lpeak)+1 (because CV is the last _white_ pixel, so add 1 to get the first _black_ pixel.)

[Chris Garry]

Thanks. This makes much more sense now. However, as Ashish suggested below, it seems like a polynomial curve fit might better assist me in finding a local minimum rather than doing the manual check you whipped up. Of course, I understand that you probably recommend some tweaking if I were to use the manual check. What do you think?

[Walter Roberson]

The only reason I can think of at the moment why you might want to do curve fitting would be if you have a mathematical model of the larynx as a curve that has a phase component to it and you want to do a phase correction.

Your curve looked fairly smooth in the test image. If there is some noise in it so that there are multiple peaks, then do some standard filtering to reduce the noise, such as doing median filtering on the heights before doing the logical test. Or use one of the MATLAB File Exchange peak-finding routines.

[Ashish Uthama]

The use of sum on the thresholded image is neat! I scratched a bit about the cumsum and headed down a different rabbit hole :)

Answer 2

[Ashish Uthama]

Here is one attempt using something Walter suggested. You might be able to modify this to generalize it enough for your application.

im = imread('l.png');

% assume its logical (since you mentioned a threshold)
bw = im(:,:,1)<1;

numRows = size(bw,1);
% Compute the 'length' from the left to the neck
lengths = zeros(numRows,1);
for rowInd = 1: numRows
    lengths(rowInd) = find(bw(rowInd,:),1,'first');
end

% You ought to see the bump as the local minima:
figure;plot(lengths);

% You might be able to code something logically to get the bump from
% the above (what Walter alludes to as 'fairly simple logic test'

% Since I am having fun, I am just gonna try something fancy. Try to fit a smooth polynomial to the curve, the bump would turn up as a
% residual.
p = polyfit(1:numRows,lengths',3);
fittedCurve = polyval(p, 1:numRows);
figure; plot(fittedCurve);

% see the differences
residuals = fittedCurve - lengths';
figure; plot(residuals);

% The chin is messing things up...find a way to discount it:
chinOffset = 50;
[junk bumpInd] = max(residuals(chinOffset:end));
bumpInd = bumpInd+chinOffset;

%Show on the image
imagesc(bw);colormap gray;hold on;
plot(lengths(bumpInd), bumpInd, 'R*','markerSize',10);