Image Convolution in Spatial Vs Image Convolution in Fourier

When you do convolution in the spatial domain with a 4x4 image and a 3x3 filter without zero padding(ie only taking output values when the whole filter is within the image), it would look like something in the attached picture (spatial.jpg). I am trying to do similar in the fourier domain, but I am not getting the results i expected. I would think that since I am filtering in the spatial domain with the whole filter window inside the image, I would be "skipping" some points. See my example code below

a=rand(4); %image
b=rand(3); %filter
c=zeros(2,2);
d=zeros(2,2);

 normalFilter=0;
%get norm
for i=1:1:3
     for j=1:1:3
      normalFilter=normalFilter+b(i,j);
  end
end
e=0; %top left convolution output in spatial.jpg
f=0; %top right convolution output in spatial.jpg
g=0; %bottom left convolution output in spatial.jpg
h=0; %top right convolution output in spatial.jpg

 %spatial convolution
for i=1:1:3
  for j=1:1:3
      e=e+a(i,j)*b(i,j);
  end
end
for i=1:1:3
  for j=2:1:4
      f=f+a(i,j)*b(i,j-1);
  end
end
for i=2:1:4
  for j=1:1:3
      g=g+a(i,j)*b(i-1,j);
  end
end
for i=2:1:4
  for j=2:1:4
     h=h+a(i,j)*b(i-1,j-1);
  end
end
c(1,1)=e;
c(1,2)=f;
c(2,1)=g;
c(2,2)=h;
c=c/normalFilter;

 %fourier convolution
k=fft2(a);
l=fft2(b);
d(1,1)=k(2,2)*l(2,2);
d(1,2)=k(2,3)*l(2,3);
d(2,1)=k(3,2)*l(3,2);
d(2,2)=k(3,3)*l(3,3);
m=ifft2(d);

I would expect at the end of this routine that "m" and "c" would be close to equivalent minus some floating point noise. Are the ways I am convolving in the spatial and frequency domains the same?

*CLARIFICATION* I am trying to understand the step by step process of how this would be done

[Image Analyst]

norm is a built-in function so DON'T use it as the name of your variable. Also, use conv2() instead of trying to do convolution yourself. I don't know what e, f, g, and h are - I'm having trouble following your uncommented code. Using conv2() would help us to understand it.

[DSP Student]

e,f,g,h correspond to the convolution output in spatial.jpg in the top left picture, top right picture bottom left picture and bottom right picture respectively. I am basically doing the same thing as conv2(a,b,'valid') i suppose. and in the fourier convolution section I am convolving using point wise multiplication on the some points that conv2(a,b,'valid') would return. I just want to make sure the general process is I am doing was correct instead of having a function do it for me

[Image Analyst]

Your manual way of doing convolution looks right, but I really have concerns about using fft2 on a 2-by-2 matrix. It's supposed to give you spatial frequency. The central element is supposed to be representative of the mean frequency. Once you use up that one, you have only 3 left to give frequencies. I think you've just quantized the heck out of it and there is no room do do anything meaningful. Try making it bigger, like 256 by 256. You can't do that manually like you did but I don't think you need to. Now that you know how the convolution process works, you don't need to do it yourself, you can use the built in function to do it for you. I mean once you know it, you know it. I learned it over 30 years ago but I don't do it myself still. I use the built-in function. At some point in your career you need to switch over, so why not now?

[DSP Student]

OK I just want to make sure I got the idea of what is going on. Thanks