I have sensors values in the form of multiple csv's want to read each file and if the length is less than 800 take average of each column and insert in the last of csv

following is the code i am testing on single file where as i have multiple files that have missing values i want to know how can i check the length of file and ignore first column and if length is less then 800 row get the length first and take average of each column and add average at the end of file . suppose a file have 631 rows . so i need to fill left rows by average means 169 values are filled .

1st Row 2nd 3rd 4th

--

631Row

average value

average value

--

--

--

800 Rows

i have files that contain 4 columns but i am working on 3 columns first column is ignored

Kindly help me in this regard ..

tmp=load('D:\Testing\1464882962.m_proc_gyro.csv');

if(length(tmp)~=800)
    for i=1:length(tmp)
            Imp_three=tmp(:,2:end);
            First_col=Imp_three(:,1);
            Second_col=Imp_three(:,2);
            Third_col=Imp_three(:,3);

            M1=mean(First_col);
            M2=mean(Second_col);
            M3=mean(Third_col);
            
            s1 = size(First_col,1); % Rows
            s2 = size(Second_col,2);
            s3 = size(Third_col,3);
            
            
            MArray=[M1,M2,M3];
        
    end        
            
end        

Answer 1

[Voss]

This will do that, including the first column (which you can continue to ignore):

tmp=load('D:\Testing\1464882962.m_proc_gyro.csv');

n_rows_target = 800; % extend to 800 rows, with mean of each column

n_rows = size(tmp,1);
if n_rows < n_rows_target
    mean_tmp = mean(tmp,1);
    tmp(n_rows+1:n_rows_target,:) = repmat(mean_tmp,n_rows_target-n_rows,1);
end

[shery khan]

Thanks can it take first mean from 1 to 631 and insert 632 value then it take mean 1 to 632 and insert 633 value so on please reply

[Voss]

I'm not sure what you are describing. What result would you want to see on the following data, if instead of 800, you want to extend it to 10 rows?

tmp = reshape(1:28,[],4) % 7 rows

tmp = 7×4

1 8 15 22 2 9 16 23 3 10 17 24 4 11 18 25 5 12 19 26 6 13 20 27 7 14 21 28

Here's the code from my answer, applied to that data, with target set to 10 (instead of 800):

n_rows_target = 10; % extend to 10 rows, with mean of each column
n_rows = size(tmp,1);
if n_rows < n_rows_target
    mean_tmp = mean(tmp,1);
    tmp(n_rows+1:n_rows_target,:) = repmat(mean_tmp,n_rows_target-n_rows,1);
end
disp(tmp);

     1     8    15    22
     2     9    16    23
     3    10    17    24
     4    11    18    25
     5    12    19    26
     6    13    20    27
     7    14    21    28
     4    11    18    25
     4    11    18    25
     4    11    18    25

Is that the right result? If not, please show me what it should be.

[shery khan]

i have analyze your code it will give average value and then same average value is inserted below but my need is different . That it can take average value from 1-10 then paste result in 11 and then take average 1-11 and paste it in 12 similarly it will fill all the missing rows like that

[Voss]

Consider the fact that including new data that is the average of the existing data doesn't change the average of the overall data:

x = 1:9

x = 1×9

1 2 3 4 5 6 7 8 9

mx = mean(x)

mx = 5

x(end+1) = mx

x = 1×10

1 2 3 4 5 6 7 8 9 5

mx = mean(x)

mx = 5

x(end+1) = mx

x = 1×11

1 2 3 4 5 6 7 8 9 5 5

mx = mean(x)

mx = 5

x(end+1:end+100) = mx

x = 1×111

1 2 3 4 5 6 7 8 9 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

mx = mean(x)

mx = 5

The average is going to stay at 5 no matter how many 5's you append to the end of it.

Therefore, the method in my answer is equivalent to your method.

If you disagree or aren't sure about that, I ask you again to please show me what the result should be in this case:

tmp = reshape(1:28,[],4)

tmp = 7×4

1 8 15 22 2 9 16 23 3 10 17 24 4 11 18 25 5 12 19 26 6 13 20 27 7 14 21 28

n_rows_target = 10; % extend to 10 rows, with mean of each column
n_rows = size(tmp,1);
if n_rows < n_rows_target
    mean_tmp = mean(tmp,1);
    tmp(n_rows+1:n_rows_target,:) = repmat(mean_tmp,n_rows_target-n_rows,1);
end
disp(tmp);

     1     8    15    22
     2     9    16    23
     3    10    17    24
     4    11    18    25
     5    12    19    26
     6    13    20    27
     7    14    21    28
     4    11    18    25
     4    11    18    25
     4    11    18    25

That is, how would those last 3 rows be different using your method?

mean(tmp(1:7,:),1)

ans = 1×4

4 11 18 25

mean(tmp(1:8,:),1)

ans = 1×4

4 11 18 25

mean(tmp(1:9,:),1)

ans = 1×4

4 11 18 25

mean(tmp(1:10,:),1)

ans = 1×4

4 11 18 25

[Image Analyst]

This answer works for one specific file. You said "i have multiple files" so to solve that part, make sure to check my answer below where I direct you to the FAQ.

Since I didn't see looping over multiple files in your code, I think maybe you couldn't figure out how to adapt it, so I'll do it for you below.

% Specify the folder where the files live.
myFolder = 'D:\Testing';
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isfolder(myFolder)
    errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder);
    uiwait(warndlg(errorMessage));
    myFolder = uigetdir(); % Ask for a new one.
    if myFolder == 0
        % User clicked Cancel
        return;
    end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.csv'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
% Loop over all CSV files in the folder.
for k = 1 : length(theFiles)
    baseFileName = theFiles(k).name;
    fullFileName = fullfile(theFiles(k).folder, baseFileName);
    fprintf(1, 'Now reading %s\n', fullFileName);
    % Now read it in as an array with readmatrix()
    thisMatrix = readmatrix(fullFileName);
    % Now process thisMatrix as you have been doing.
end

But please, still remember to check out the FAQ because there is lots of other good information in there.

[shery khan]

Thank you so much for help

Answer 2

[Image Analyst]

See the FAQ:

https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files?