You have several options.
1) If every cell in your cell array contains a scalar double value, cell2mat() will work properly. You probably still have other data types mixed into your cell array.
2) You can get a numeric array directly from xlsread by using xlsread with this syntax: num = xlsread('filename.xls');
3) Instead of xlsread, try using readtable('filename.xls'). This will return a table instead of a cell array or double array.