How can I conduct statistical analysis of runs that collate into one variable?

I am conducting an experiment where all my model runs collate into one variable. i.e. my runs are: Temp001 Temp002 Temp003 etc. To view the data for each of these runs I have to load Temp001 into the workspace as variable Temp and it displays the data. If I want to view the data for Temp002, I have to do the same and it loads the data for Temp002 into Temp as well. I cannot view the data for Temp001 and Temp003 at the same time as they are always loaded into the single variable 'Temp'.

For example:

Temp001: 4 5 6 7

Temp002: 1 2 3 4

Temp003: 9 8 7 6

load Temp001

display(Temp): 4 5 6 7

load Temp002

display(Temp): 1 2 3 4 % I can no longer view the data for Temp001, only for Temp002

I am conducting hundreds of runs and want to conduct statistical analysis on the runs (median, percentiles etc). How do I do this without having to save each individual run (of which there are hundreds) as separate variables?

For example:I want to find the median of each column over the 3 runs. Is there a way I can do this without having to type (the below) hundreds of times?:

load Temp001

A=Temp

load Temp002

B=Temp

load Temp003

C=Temp

D=median(A:C)

Please help me, it will take forever otherwise!

[dpb]

What does a "model run" consist of? The problem begins there in that you're not creating a way to easily handle multiple cases there.

Show us how that is done and it's probably trivial (or nearly so) to fix...

A big klew one isn't doing things properly in Matlab is when start generating sequential variables with subsequent letter or number as above...

Answer 1

[Image Analyst]

It's covered by the FAQ: https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

For example, it might be close to this:

% Specify the folder where the files live.
myFolder = 'C:\Users\yourUserName\Documents\My Data';
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isdir(myFolder)
  errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
  uiwait(warndlg(errorMessage));
  return;
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, '*.mat'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1 : length(theFiles)
    baseFileName = theFiles(k).name;
    fullFileName = fullfile(myFolder, baseFileName);
    fprintf(1, 'Now reading %s\n', fullFileName);
    % Now do whatever you want with this file name,
    % such as taking the median
    s = load(fullFileName); % Load contents of file into structure s.
    thisData = s.Temp;
    theMedians(k) = median(thisData);
    theMeans(k) = mean(thisData);
end

[dpb]

There's that, too, IA, and I thought of suggesting precisely the same route -- but thought I'd suggest seeing if OP couldn't just do the whole thing from the git-go by rewriting the original model just a little...

[Elizabeth Hearn]

Unfortunately I cannot rewrite the code as it hasnt come from me.

This does something but it doesnt quite work. Each run is in the current folder under 'Temp001, Temp002, Temp003 etc' but the variable 'Temp' is in the working directory. Therefore in this part:

filePattern = fullfile(myFolder, '*.mat'); % Change to whatever pattern you need.

when I replace the * with 'Temp' it doesn't recognise it as it isn't Temp that is in the folder path, it is the 'Temp00x'. For example it gives this output:

theFiles =

0×1 empty struct array with fields:

name

folder

date

bytes

isdir

datenum

[Image Analyst]

If you can't change the code, then you're stuck with what you have. However if you don't change anything, then the situation will remain as it is.

[dpb]

Well, even if you didn't write the actual application, you should be able to write a wrapper routine that calls it in a loop...

The pattern IA means is to match whatever is the needed filename extension and wildcard to match the naming convention your files use-- load uses '.mat' by default so that is the pattern he illustrated and is probably the correct one. I'd probably have been a little more selective and written 'Temp*.mat', just in case there are any other .mat files in this subdirectory, to exclude anything but the specific ones wanted.

The subdirectory string needs to be the other argument to fullfile() where IA gave you the hint to put it by using "myFolder" as the placeholder.

The result of executing fullfile(myFolder, '*.mat') needs to give you a proper either fully-or partially qualified filename wildcard string that refers to the desired folder.