Converting loop to vector operation

2 views (last 30 days)
Azura Hashim
Azura Hashim on 8 Jul 2017
Edited: Jan on 9 Jul 2017
Hi,
I have a large dataset where I calculate the value of some result in each row based on data in preceding rows. Currently it is taking too long due to the large data. I have included a simplified example below. Can this be converted to a vector operation? Alternatively and less preferably can this be done in parallel? Thank you.
MATLAB code
dataset=table();
dataset.assetname=[{'st1'};{'st1'};{'st1'};{'st2'};{'st2'}];
dataset.time=[1;2;5;2;3];
dataset.price=[1.1;1.2;1.1;2.1;2.2];
dataset.quantity=[10;15;5;20;25];
dataset.ordertype=[{'buy'};{'buy'};{'sell'};{'buy'};{'sell'}];
dataset.last_buy_quantity_price_x=repmat(NaN,height(dataset),1);
price_x=1.1;
for row=1:height(dataset)
filteredrows=strcmp(dataset.assetname(row),dataset.assetname) & dataset.time(row) >=dataset.time;
lastrow=max(find(dataset.price(filteredrows)==price_x & strcmp(dataset.ordertype(filteredrows),'buy')));
if isempty(lastrow)
tempvar=0;
else
tempvar=dataset.quantity(lastrow);
end
dataset.last_buy_quantity_price_x(row)=tempvar;
end

Accepted Answer

Jan
Jan on 8 Jul 2017
Edited: Jan on 8 Jul 2017
I don't think that you can vectorize this. But an acceleration is possible:
lastrow = max(find(dataset.price(filteredrows)==price_x & ...
strcmp(dataset.ordertype(filteredrows), 'buy')));
% Replacement:
lastrow = find(dataset.price(filteredrows)==price_x & ...
strcmp(dataset.ordertype(filteredrows), 'buy')), 1, 'last');
And:
dataset.last_buy_quantity_price_x = zeros(height(dataset), 1);
...
if ~isempty(lastrow)
dataset.last_buy_quantity_price_x(row) = dataset.quantity(lastrow);
end
end
Compare the ordertype with 'buy' once only before the loop:
isBuy = strcmp(dataset.ordertype, 'buy');
Then inside the loop:
lastrow = find(dataset.price(filteredrows)==price_x & isBux(filteredrows), 1, 'last');
The same works with "dataset.price == price_x" also.
What about removing all or(dataset.price ~= price_x, ~strcmp(dataset.ordertype, 'buy') before the loop? This might reduce the data set massively:
price_x = 1.1;
m = or(dataset.price ~= price_x, ...
~strcmp(dataset.ordertype, 'buy'));
name = dataset.assetname(m);
time = dataset.time(m);
price = dataset.price(m);
quantity = dataset.quantity(m);
last_buy = zeros(numel(name),1);
for row = 1:numel(name)
filtered = strcmp(name(row), name) & (time(row) >= time);
lastrow = find(filtered, 1, 'last');
if ~isempty(lastrow)
last_buy(row) = quantity(lastrow);
end
end
dataset.last_buy_quantity_price_x = zeros(height(dataset), 1);
dataset.last_buy_quantity_price_x(m) = last_buy;
This saves teh time for addressing the struct fields also and I think the simpler the code looks like, the easier is it to maintain.
  2 Comments
Azura Hashim
Azura Hashim on 9 Jul 2017
Thanks Jan, much appreciated. I will try these out. Is it also possible to parallelize the calculation?
Jan
Jan on 9 Jul 2017
Edited: Jan on 9 Jul 2017
I assume so. Did you try it?
If you provide some real world input data, experimenting with the code would be much easier. It matters e.g. if the data are sorted.

Sign in to comment.

More Answers (0)

Categories

Find more on Programming in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!