vectorizing a script with cellfun

I'm aiming to import data from various folder and text files into matlab.

clear all
main_folder = 'E:\data';
    %Directory of data
TopFolder = dir(main_folder);
    %exclude the first two cells as they are just pointers. 
TopFolder = TopFolder(3:end);
TopFolder = struct2cell(TopFolder);
Name1 = TopFolder(1,:);
    %obtain the name of each folder
dirListing = cellfun(@(x)dir(fullfile(main_folder,x,'*.txt')),Name1,'un',0);
Variables = cellfun(@(x)struct2cell(x),dirListing,'un',0);
FilesToRead = cellfun(@(x)x(1,:),Variables,'un',0);
    %obtain the name of each text file in each folder

This provides the name for each text file in each folder within 'main_folder'. I am now trying to load the data without using a for loop (I realise that for loops are sometimes faster in doing this but I'm aiming for a compact script).

The method I would use with a for loop would be:

for i = 1:length(FilesToRead);
    data{i} = cellfun(@(x)dlmread(fullfile(main_folder,Name1{i},x)),FilesToRead{i},'un',0);
end
[~,Variable] = cellfun(@(x)fileparts(x),FilesToRead{1},'un',0);

Is there a method which would involve not using loops at all? something like cellfun within cellfun maybe? I also realise that textscan is better for importing text files, but dlmread works fine in this example.

In addition is it possible to create a variable in the workspace corresponding to 'Variable'?

Answer 1

[Andrei Bobrov]

data = arrayfyn(@(ii)cellfun(@(x)dlmread(fullfile(main_folder,Name1{ii},x)),FilesToRead{ii},'un',0),1:length(FilesToRead),'un',0);

Answer 2

[Sean de Wolski]

It looks like you could wrap cellfun around that instead of a for-loop.

I'm obligated to remind you that well written for-loops will likely be faster and easier to read. I don't completely understand why you want to use 'compactness' as a metric for code quality. Most people will understand what two for-loops do. It takes someone with a fair amount of MATLAB experience to understand cellfun

Answer 3

[Jan]

cellfun with anonymous functions is slow. I'm convinced, that a simple FOR-loop approach is faster - concerning programming, debug and runtime.

Omitting the first two entries of the reply of dir is not secure, because it is not documented, that . and .. are set to the front. Although I did not see them appear at another location, I use strcmp({TopFolder.name}, '.') to exclude these folders.

[Richard]

when would you suggest to avoid loops? also whatdo you mean by stating 'cellfun' with anonymous function?

[Jan]

@lestyn:

cellfun(@fileparts, a, 'un', 0) is faster than

cellfun(@(x) fileparts(x), a, 'un', 0).

The later is CELLFUN with an anonymous function.