Your code first puts the successive integers from 1 to n*m in the vector 'invKey'. Then it subjects them to a randomly determined permutation as give by 'Key'. This has the effect of getting the inverse of the Key value. That is, suppose n*m = 6, and suppose 'randperm' yields Key = [4,2,1,5,6,3]. Then invKey is initially [1,2,3,4,5.6]. After the invKey(Key)=invKey step, invKey will contain [3,2,6,1,4,5] which is the inverse of Key.