Optimizing a strfind operation for speed
Show older comments
Hi Matlab Central,
I am an inexperienced programmer looking to speed up the code I have. I know enough to go into profiler and look at what is taking a long time, and I think it is this bit here:
UniqueTFArray=unique(CombinedArray);
TFtable=zeros(size(AAA,1), length(UniqueTFArray));
for i=1:size(AAA, 1)
for j=1:length(UniqueTFArray)
TFtable(i,j)=~isempty(strfind(AAA.Regulator{i,1}, UniqueTFArray{1,j}));
end
end
TFSum=sum(TFtable);
figure; bar(TFSum);
AAA is a few thousand long, and UniqueTFArray is a few hundred, so the way I have it written, I think the profiler is telling me it gets called like 520,000 times so it is slow.
Now, I have a few ideas that I think could be of use.
Most of AAA.Regulator is empty, so length is 0. Should I put the strfind line in an if statement and only call it if the length is greater than 0? That would save time I think...
Or is there a fundamentally better approach?
Thank you very much!
Accepted Answer
Bjorn Gustavsson
on 17 Jan 2014
Maybe you'd get some speedup by using strfind on the entire AAA.Regulator cell array at once and then handle the result of those matches separately:
for i1 = 1:length(UniqueTFArray)
TempStrFindRes{i1} = strfind(AAA.Regulator, UniqueTFArray{1,i1});
end
for i2 = 1:length(TempStrFindRes)
TFtable(:,i2) = 1 - cellfun(@isempty,TempStrFindRes{i2});
end
If that's approximately what you want then at least it reduces the repeated overhead repeated calls to strfind would cause.
HTH
2 Comments
Sarutahiko
on 21 Jan 2014
Edited: Sarutahiko
on 21 Jan 2014
Bjorn Gustavsson
on 21 Jan 2014
Have you tried my suggestion? Did it work? My simplistic test with guessed parameters indicated that the results were identical, but couldn't be bothered to make as large arrays to test with, so no timing of the 2 versions.
The difference is that your double loop calls strfind length(AAA.Regulator)*length(UniqueTFArray) number of times looking for a match. However strfind can handle cell-arrays of strings in its first input argument, so that's what I utilize. For each element in UniqueTFArray I look for matches in all of AAA.Regulator so my version only calls strfind length(UniqueTFArray) number of times. This produces a cell array of outputs that is then combined in the second loop. The tradeoff is that the number of calls to strfind is reduced in my version with the downside of having to call cellfun in the second loop instead. The question is which version causes more overheads...
HTH
More Answers (2)
Walter Roberson
on 17 Jan 2014
0 votes
There is no point in searching for a string that will never be found.
Question: is AAA.Regulator only unique words, or are you ending up searching multiple times for some words?
Do I understand correctly that the point of the code is to count the number of times that each word of a corpus of words appears in each subset? And to check, are you looking for exact matches, a whole word matching a whole word, or are you looking for the case where the words in AAA.Regulator{i,1} appears anywhere within any of the words? For one thing, if you are looking for whole-word matches then you can "break" out of the "for j" loop as soon as a match occurs.
3 Comments
Sarutahiko
on 17 Jan 2014
Edited: Sarutahiko
on 17 Jan 2014
Walter Roberson
on 17 Jan 2014
A sample AAA.Regulator entry and a sample entry from UniqueTF would help.
Sarutahiko
on 21 Jan 2014
Sarutahiko
on 21 Jan 2014
0 votes
Categories
Find more on Data Type Identification in Help Center and File Exchange
Products
on 17 Jan 2014
on 21 Jan 2014
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
