How should Fisher Vector be represented for image classification?

Hello everyone,

Currently I'am working with Image Classification. I just wonder how we are going to implement Fisher Vector using vlfeat library to represent the extracted SURF local feature. The implementation of hard voting using k-means is straight forward, but fisher using GMM to construct visual word. Below is the code:

     I = imread('2296.jpg'); 
     a=rgb2gray(I);
     points = detectSURFFeatures(a); 
     [feates, valid_points] = extractFeatures(a, points);
     numClusters = 50 ;
     [means, covariances, priors] = vl_gmm(feates, numClusters);
     encoding = vl_fisher(feates, means, covariances, priors)

The final output in encoding produce 60300 X 1 matrix(different image will produce different number). Then what is the next step to represent them in vector? This is because the classifier only work with the fixed length vector.

Cheers...

Answer 1

[ZHAO Ling]

I have the same problem,have you solved? I think, the input of GMM and Fisher vector should not the same. cause the GMM is training a codebook, so need more input rather than just one. I am thinking train all the images with GMM to generate a code book, and then fisher vector will create for each images in "words" according to the codebook. Then I have doubt about how to train the GMM with all the images together? one by one and saved all the means, priors, variances into a gaint matrix? Someone can help please?

[Erysham]

Yes. The only thing you need to do is to transpose the matrix of local feature. for instance, originally SIFT will generate 1 * 128 feature vector. Now you have to transpose it to 128 * 1 before you can feed into GMM. The final feature vector will be calculate as 128 * K * 2 where K=no of gaussian. So all image will have the same length of feature vector.

cheers...

[Erysham]

You have to use all images in the same category to perform GMM to produce means, covariance and priors. Then, the vl_fisher should be perform on a single image by referring the means, covariance and priors produced by GMM.