Removing columns if single value is more than threshold

I have a very big data matrix which I am trying to filter a bit. I would like to remove whole columns, if any value is for example 10% greater or less than row average. I checked the rmoutliers function, but I don't know how I can make that work the way I need. Another matter is that some columns are fine for my use, but they are scaled up, so they would get probably filtered out too with that method. That is fine, but could that be avoided by first normalizing the data somehow, and then restoring the filtered data to original scale after that. I would appreciate the help very much

Answer 1

[Adam Danz]

" I would like to remove whole columns, if any value is for example 10% greater or less than row average"

Demo:

% Create 100x10 matrix
data = rand(100,10) .* linspace(1,100,10); 

% Determine which columns have at least 1 values that is 
% within +/- 10% of the row's average
rowAverages = mean(data,2);
isNearAvg = abs(data - rowAverages) <= rowAverages  * 0.1;  % 10% threshold
replaceColumn = any(isNearAvg,1); 

% Option 1: Repalce the column with NaNs, thereby preserving the original structure
data(:,replaceColumn) = NaN

% Option 2: Remove the columns (use replaceColumn to see which cols were removed)
data(:,replaceColumn) = []

[Mat P]

I think this works well, thank you. I Didn't think the logical way.

[Adam Danz]

Glad I could help!

Functions like rmoutliers come in handy during data exploration and if you are using a well-established method of outlier removal but it's often better to write your own functions when the method requires customization.