How to use Support Vector Machine for Speech recognition

Hi,

I want to recognize silent speech words based on lip contour centroid key points .

Example : I have created MYSQL Database table column name "Words" , here 1. Hello , 2.How are You ,3. Nice to Meet You . etc.. after i have connected MySQL connection using matlab .Now i want to silent lip SVM(Support Vector Machine Speech Recognition) isolated word Pattern Matching? here reference link which i am following : http://www.ee.oulu.fi/~gyzhao/Download/Databases/OuluVS/lipreading-final-DC.pdf if any solution let me know .

Thanks

[SAMEER ahamed]

How word Image Sequences can performed using HMM? With the reference link : lip image sequence , please let me know any idea how i can extract word's ?

[Walter Roberson]

It sounds as if you are asking is to read the papers and explain them to you. That is not something that we, as volunteers, do. You resolve the algorithm, and start implementing, and we help you with MATLAB library calls or language features.

[SAMEER ahamed]

Hi,

Thank's for reply me , I wan't to implement lip reading extract visual speech , so could you please let me know , after detection lip shape ,center points , next step algorithm ? I have confusion about visual feature ,Support Vector Machine and HMM model ?

[Walter Roberson]

This forum is about MATLAB: the language features, the library calls, the toolkits, interfacing to MATLAB, using MATLAB to interface to other things, decoding MATLAB error messages, speeding up code, and the like. All things that are specific to MATLAB.

This forum is not about algorithms. From time to time, a volunteer will suggest a better algorithm than what was proposed in the discussion, but the rest of the time people are expected to select the algorithm that they will use and then the volunteers help out when the person runs into problem implementing the algorithm. If the person picks an inefficient algorithm or one that does not produce top quality results, then the volunteers help the person with what was chosen, not with what could have been chosen.

If you do not know which algorithm to use, then you should read more papers, or find a relevant resource on the Internet, or talk to a professor or a teachers assistant or a tutor or a supervisor or use internal resources at your workplace to find someone at your workplace that can assist.

Or you can hire a consultant with experience in that field. Or you can hire a consultant who has experience in studying science papers and sifting through Internet resources and understanding scientific material in general, with the consultant evaluating the choices and making recommendations to you and teaching you about the topic.

I volunteer here about MATLAB, but I do not volunteer here to research algorithms and make recommendations. I can be hired to do research, though.

[SAMEER ahamed]

Thank's for reply me ,I have got research paper exact my task related about based on lip movement recognize word . SVM Phrases , below code i have mentioned not getting binary lip image if any idea let me know result image previous forum ?

YIQ = rgb2ntsc(hue1);
     Y1 = YIQ(:,:,1);
    I1 = YIQ(:,:,2);
    Q = YIQ(:,:,3);

      J = imadjust(Y1);
      t = graythresh(blue_Image);
  imCoin = (red_Image > hueThresholdHigh);
      K = medfilt2(red_Image);
      [level1 EM]= graythresh(K);
      BW_lip = im2bw(red_Image);
      Binary_Hue = red_Image > T;
      level = graythresh(J);
     binary_Image = im2bw(J,level);
     [imx,imy]=size(binary_Image);
     bw_Fill= imfill(binary_Image,'holes');
     labeledImage = bwlabel(bw_Fill, 8);
     blobMeasurements = regionprops(labeledImage, 'area');
     areas = [blobMeasurements.Area];
     [maxArea largestBlobIndex] = max(areas);
      bw_area = bwareaopen(bw_Fill ,2500);
     BWnobord = imclearborder(bw_area, 4);
     seD = strel('disk',1);
  %     BWfinal = imerode(BWnobord,seD);
      BWfinal = imerode(BWnobord,seD);
      BWoutline = bwperim(bw_area);
      Segout = mouthcrop;
      Segout(BWoutline) = 255;

[Walter Roberson]

You have not defined blue_Image or red_Image or hueThresholdHigh

[SAMEER ahamed]

i am not getting lip hue region shape. so far i have applied Adaptive threshold algorithm also ,but i wouldn't get exact lip shape ?

if true
mouthcrop = imcrop(mregcrop ,[x3 y3 w3 h3]);
red_Image = mouthcrop(:,:,1);
blue_Image = mouthcrop(:,:,2);
hueThresholdHigh = graythresh(red_Image); 
imCoin = (red_Image > hueThresholdHigh);
end

Answer 1

[Image Analyst]

I'm sorry but to me this code looks totally random and disorganized, like you just threw together a bunch of unrelated chunks of code and hope they would somehow magically work together. Taking a (supposed) hue image and converting to YIQ color space? Huh? Taking both the Y and the blue channels and automatically thresholding them? Huh? Then taking the red channel and manually thresholding it? What? Calling imadjust on the Y channel? Totally unnecessary. Then threshold that? What? Measuring the filled binary image but then never doing anything with the results? Why? Then doing a bunch of morphological stuff on a binary image that came originally from the red channel and burning it into the original image. Why not just use bwboundaries and plot to put the outline into the overlay? Okay, that's just a jumble of code with no organized algorithm behind it.

Sorry to be so harsh but there is so much wrong with this algorithm that it isn't worth trying to fix. Like Walter said, we don't do algorithm development in this forum. I suggest you try to follow the algorithm in the paper rather than the one you listed here, which looks nothing like that.

One tip I can give is that for skin, the red channel has the least contrast between kips and surrounding skin, so you you probably shouldn't be using the red channel, at least not by itself.

[SAMEER ahamed]

Thanks for reply me , now i have modify my below code , but i am not getting exact shape where i have make mistake ?.

if true
  % code
hue1 = rgb2hsv(mouthcrop);
  YIQ = rgb2ntsc(hue1);
   Y1 = YIQ(:,:,1);
  I1 = YIQ(:,:,2);
  Q = YIQ(:,:,3);
  J = imadjust(Y1);
  level = graythresh(J);
 binary_Image = im2bw(J,level);
 bw_Fill= imfill(binary_Image,'holes');
  bw_area = bwareaopen(bw_Fill ,2500);
      bw_edge = bwlabel(bw_area, 8);
       blobMeasurements = regionprops(bw_edge, 'BoundingBox', 'Centroid','Area');
        numberOfBlobs = size(blobMeasurements, 1);
   subplot(3, 3, 9); imagesc(mouthcrop);
  title('Lip Contour Key Points'); axis square;
       hold on;
      boundaries = bwboundaries(bw_area);  
      numberOfBoundaries = size(boundaries);
      for k = 1 : numberOfBoundaries
          thisBoundary = boundaries{k};
          plot(thisBoundary(:,2), thisBoundary(:,1), 'r', 'LineWidth', 2);
      end
      hold off;
   numberOfBlobs = size(blobMeasurements, 1);
      for k = 1 : numberOfBlobs
          blobCentroid = blobMeasurements(k).Centroid; 
     end
      centroidColumn = int32(blobCentroid(1));% "X" value
      centroidRow = int32(blobCentroid(2)); %"Y" value.
      middleColumn = bw_Fill(:, centroidColumn);
      topRowY = find(middleColumn, 1, 'first'); % Find top.
      bottomRowY = find(middleColumn, 1, 'last'); % Find bottom.
      middleRow = bw_Fill(centroidRow,:);
      leftColumnX = find(middleRow, 1, 'first'); % Find left.
      rightColumnX= find(middleRow, 1, 'last'); % Find right.
      line(centroidColumn, centroidRow, 'Marker', '*', 'MarkerEdgeColor', 'b');
      line(centroidColumn, topRowY, 'Marker', '*', 'MarkerEdgeColor', 'g');
      line(centroidColumn, bottomRowY, 'Marker', '*', 'MarkerEdgeColor', 'g');
      line(leftColumnX, centroidRow, 'Marker', '*', 'MarkerEdgeColor', 'g');
      line(rightColumnX, centroidRow, 'Marker', '*', 'MarkerEdgeColor', 'g');
end

[Image Analyst]

You forgot to attach your script and image.

[SAMEER ahamed]

Here out put result i'm getting ,if any changes required to get exact lip shape boundary ?.

[Image Analyst]

More light is the best solution. Lack of light is giving no contrast to these lips.