If statement does not work

I have a problem in populating correctly a table. Starting from A(mx4) and B(nx4), where m>n, I have to create C(mx4). According to the code whether the if statement it is not true 4 coloumn have default value.

function [Int12] = bundle(F1,F2)
    
    A= zeros(height(F2), 8);
    Int12= array2table(A);
    Int12.A2 = NaT(length(A),1,'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );
    Int12.A6 = NaT(length(A),1, 'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );
    
    j=1;
    for i=1: height(F2)
        
        Int12(i,5)= F2(i,2); % tag
        Int12(i,6)= F2(i,1); % timestamp
        Int12(i,7)= F2(i,3); % X
        Int12(i,8)= F2(i,4); % Y
        
        
        if abs(F2{i,1}- F1{j,1})<= duration('00:00:01.000000000')
            
            Int12(i,1)= F1(j,2); % tag
            Int12(i,2)= F1(j,1); % timestamp
            Int12(i,3)= F1(j,3); % X
            Int12(i,4)= F1(j,4); % Y
            j=j+1;
            
        end
    end
end

The problem is that the function works but at a certain point even if the if statement is true it does not work anymore.

[the cyclist]

Can you upload the workspace data in a MAT file, such that we can run your code and see what is happening? It's a bit confusing, since you mention variables A, B, and C ... which do not appear in your code at all.

[Andrea Sbaragli]

function [Int12] = bundle(F1,F2)
    
    A= zeros(height(F2), 8);
    Int12= array2table(A);
    Int12.A2 = NaT(length(A),1,'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );
    Int12.A6 = NaT(length(A),1, 'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );
    
    j=1;
    for i=1: height(F2)
        
        Int12(i,5)= F2(i,2); % tag
        Int12(i,6)= F2(i,1); % timestamp
        Int12(i,7)= F2(i,3); % X
        Int12(i,8)= F2(i,4); % Y
        
        
        if abs(F2{i,1}- F1{j,1})<= duration('00:00:01.000000000')
            
            Int12(i,1)= F1(j,2); % tag
            Int12(i,2)= F1(j,1); % timestamp
            Int12(i,3)= F1(j,3); % X
            Int12(i,4)= F1(j,4); % Y
            j=j+1;
            
        end
    end
end

[the cyclist]

Thanks for posting the data.

Can you please do two more things, to help us help you?

• Could you be very specific about where the output is giving you something you don't expect, and tell us what you did expect, and why? It is too vague for you to say, "at a certain point even if the if statement is true it does not work anymore"
• Could you post a small subset of the data that exhibits the problem? Your current code takes 3 minutes to run on my machine, and it would be faster to debug something smaller.

[Andrea Sbaragli]

1st - Even if the statement if true the code at a certain point does not enter on it anymore. Picture below

In order to give you a a quick idea you could check manually by yourself

F2(79329,1) - F1(47416,1) <= 1 sec

but if you check on Int12(79329,1:4) values are either 0 or NaT

2nd- I try and then I post immediately the subset

Let me know please if you understand the first point please

[Andrea Sbaragli]

Regarding the subset I did the following:

F1(1:43285,:)=[]; % F1= 4329x4
F2(1:74000,:)=[]; % F2=5329x4

Now the error is changed and is:

Row index exceeds table dimensions. in abs(F2{i,1}- F1{j,1})<= duration('00:00:01.000000000')

[the cyclist]

I understand. Here is what I notice:

Your code never actually compares F2(79329,1) and F1(47416,1). When i == 79329, the value of j is 4106. Therefore, you only compare F2(79329,1) and F1(4106,1). That difference is not less than 1 second.

I don't know if you are incrementing j, and making comparisons, in the way you intend.

[Andrea Sbaragli]

Yeah that's true j at the end of the running is equal to 4106 but why?

I mean as it stands the problem occurrs because j at a certain point, in this case once it is equal to 4106, it is not incremented... but that's weird since it is coded .

j=j+1; 

[Andrea Sbaragli]

Moreover, I think conceptually is correct. SInce the size of F1 is less than F2 the for loop, i-indexed, is done in the latter. At this point, whenever the comparison is true since j is the index of F1, j is incremented by one....

Of course, in order to not loose any data Int12 has the same number of rows of F2 and therefore indexed in i. :(((

[the cyclist]

What you intend your code to do is not perfectly clear to me.

Here is what I see it doing:

1. Go systematically down the rows of F2, until the time is within 1 second of the first row of F1 (because j==1).
2. The first time this happens is when i==31014.
3. Fill the row 1 values of F1 into row 31014 of Int12
4. Next, continue along F2 (from row i==31015), until the time is within 1 second of the second row of F1 (because now j==2)
5. Because F2(31015,1) and F1(2,1) are within 1 second of each, fill the row 2 values of F1 into Int12(31015)
6. ... and so on

You can already see that all rows of F2 will not be compared with all rows of F1, as you seem to expect.

[Andrea Sbaragli]

Yes that's right.

For this purpose I have modified the code as it follows but, as you probably now better than me, it is too expensive computationally...

function [Int12] = bundle(F1,F2)

A= zeros(height(F2), 8);
Int12= array2table(A);
Int12.A2 = NaT(length(A),1,'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );
Int12.A6 = NaT(length(A),1, 'Format', 'dd-MMM-yyyy HH:mm:ss.SSSSSSSSS' );

j=1;

for i=1: (height(F2))

    Int12(i,5)= F2(i,2); % tag
    Int12(i,6)= F2(i,1); % timestamp
    Int12(i,7)= F2(i,3); % X
    Int12(i,8)= F2(i,4); % Y
   
       for j= 1: height(F1)
        if abs(F2{i,1}- F1{j,1})<= duration('00:00:01.000000000')

             Int12(i,1)= F1(j,2); % tag
             Int12(i,2)= F1(j,1); % timestamp
             Int12(i,3)= F1(j,3); % X
             Int12(i,4)= F1(j,4); % Y
             

        end
       end
    


end




end

[the cyclist]

It is no surprise to me that your code is computationally expensive. You are comparing 79329 elements to 47614 elements. That is simply a large number of comparisons.

Also, if all of your times were close to each other, you would be creating a table that is over 3 billion rows long (79329x47614). I don't know how big it will actually be, but I expect it is much larger than the memory you have preallocated.

You haven't really described the overall intent of your code. Do you really need to store every instance where a row of F2 is close in time to F1? Or maybe your output is actually something simpler and smaller.

What does the result need to be? It is possible that there is a faster way to do it.