For loop optimization help for working with large data

1 view (last 30 days)
I am working with a data set collected from a wearable wristband that tracks phsyiological data and has a button that creates a data point with the posix time when it was pressed. We are having participants indicate increases by pressing the button two times consecutively and one standalone press indicates a decrease. Now I am trying to perform a vector that goes through each time table and assign a running total that is synchronized to the data collection times (ex. if an increase event occurs at 8AM everything before then should be whatever the previous value. At 8AM the running total would increase by 1 and carry that total until adjusted by the next tagged event and so on and so forth)
Right now I am using a for loop that goes through each row and sees if the time of each physiological data point is between each tagged time point and then I assign that a value based on the already calculated running total variable.
y= numel(TEMPX);
TEMPtry = randi(10,y,1);
for l = 1:numel(TEMPX)
if TEMPX.Time(l) <= Tagstime(1,1)
TEMPtry(l,1)= 0;
continue
end
if TEMPX.Time(l) > Tagstime(1,1) && TEMPX.Time(l) < Tagstime(2,1)
TEMPtry(l,1) = total(1,1);
continue
end
if TEMPX.Time(l) >= Tagstime(2,1) && TEMPX.Time(l) < Tagstime(3,1)
TEMPtry(l,1) = total(2,1);
continue
end
if TEMPX.Time(l) >= Tagstime(3,1) && TEMPX.Time(l) < Tagstime(4,1)
TEMPtry(l,1) = total(3,1);
continue
end
if TEMPX.Time(l) >= Tagstime(4,1) && TEMPX.Time(l) < Tagstime(5,1)
TEMPtry(l,1) = total(4,1);
continue
end
if TEMPX.Time(l) >= Tagstime(5,1) && TEMPX.Time(l) < Tagstime(6,1)
TEMPtry(l,1) = total(5,1);
continue
end
if TEMPX.Time(l) >= Tagstime(6,1) && TEMPX.Time(l) < Tagstime(7,1)
TEMPtry(l,1) = total(6,1);
continue
end
if TEMPX.Time(l) >= Tagstime(7,1) && TEMPX.Time(l) < Tagstime(8,1)
TEMPtry(l,1) = total(7,1);
continue
end
if TEMPX.Time(l) >= Tagstime(8,1) && TEMPX.Time(l) < Tagstime(9,1)
TEMPtry(l,1) = total(8,1);
continue
end
if TEMPX.Time(l) >= Tagstime(9,1) && TEMPX.Time(l) < Tagstime(10,1)
TEMPtry(l,1) = total(9,1);
continue
end
if TEMPX.Time(l) >= Tagstime(10,1)
TEMPtry(l,1) = total(10,1);
continue
end
end
beep on
*Note: this is from a sample data set I was working from where there only ended up that having 10 tagged items. In an actual data set the number of tagged items could vary with no defined maximum. "TEMPX" is a timetable, "Tagstime" is a datetime variable, "total" is a double
As you could imagine this script takes FOREVER to complete when y = 342056 since the sample rate is so high and I am currently using a "for loop" method. Can someone help me come up with a better way to achieve this operation? I looked at some other posts that recommended vectorization but I am not sure how to apply that method to achieved the desired outcome.
desired output: Tags [8:00:01; 8:00:02; 12:00:01; 13:01:01; 13:01:02], total [1,0,1],
TEMPtry(starttime: 8:00:00, 1) = 0
TEMPtry(8:00:01:12:00:00, 1) = 1
TEMPtry(12:00:01:13:00:00, 1) = 0
TEMPtry(13:00:01: end, 1) = 1
*I know that's not the correct notation but hopefully that gives you an idea of what I'm trying to do.
  4 Comments
Catalytic
Catalytic on 28 Jan 2022
Yes, but it would help us more if you put all of your variables in a single .mat file instead of 3 separate ones. That way there is less for us to download/load.
Cameron Johnson
Cameron Johnson on 28 Jan 2022
Thanks for the suggestion. That makes way more sense to do it like this

Sign in to comment.

Answers (1)

Matt J
Matt J on 28 Jan 2022
Edited: Matt J on 29 Jan 2022
load('Example Workspace.mat')
edges=[seconds( Tagstime-Tagstime(1) );inf];
T=seconds( TEMPX.Time-Tagstime(1) );
bin=discretize(T , edges);
inside=~isnan(bin);
TEMPtry = zeros(numel(TEMPX),1);
TEMPtry(inside)=total(bin(inside));
plot(TEMPtry); axis padded
  7 Comments
Cameron Johnson
Cameron Johnson on 29 Jan 2022
Edited: Cameron Johnson on 2 Feb 2022
Actually there does seem to be one small hiccup. It only seems to work for the pairs where there is a second between them (indicating an increase) but it doesn't work well with the single button presses (indicating a decrease). If you could make some suggestions that would be extremely appreciated.
Matt J
Matt J on 3 Feb 2022
I suggest you read the documentation for discretize(),
My initial answer chose the bin edges with Tagstime rounded to seconds,
edges=[seconds( Tagstime-Tagstime(1) );inf];
but there is nothing sacred about that choice.

Sign in to comment.

Categories

Find more on MATLAB in Help Center and File Exchange

Products


Release

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!