Something wrong with "floor" or "fix" functions or it's my code?

6 views (last 30 days)
row and col contain a lot of values of rows and cols. I'm trying to scale down the rows and cols to 1/4 sizes (rs,cs) and ps is the linear index in the scaled down matrix.
rs = fix((row-1)/4) + 1; %scale down of rows
cs = fix((col-1)/4) + 1; %scale down of cols
ps = (cs - 1)*140 + rs; %linear index in 140x140 matrix
It complains that the Index must not exceed 19600. I isolated the problem whenever the col = 559, the cs = 141 instead of 140. If I run the code individually on the command window >> cs = floor((559-1)/4) + 1; or >> cs = fix((559-1)/4) + 1; it gives me the correct value 140.
Anyone sees the problem that I can't see? I appreciate your help.
DB on 10 May 2023
See attached variables: Pointers, row, col(they are int32).
Below are the calculations for row and col before the calculation above:
row = rem(Pointers-1,560) + 1;
col = (Pointers-row)/560 + 1;
Appreciate anyone who can help spot the problem.

Sign in to comment.

Accepted Answer

Stephen23 on 10 May 2023
Edited: Stephen23 on 10 May 2023
"Something wrong with "floor" or "fix" functions"
Nope, in fact those functions do absolutely nothing in your code.
"or it's my code?"
"Anyone sees the problem that I can't see?"
The problem is that your data are pure integers. MATLAB clearly tells us that they are INT32 class:
S = load('variables.mat')
S = struct with fields:
Pointers: [2038×1 int32] col: [2038×1 int32] row: [2038×1 int32]
When performing arithmetic on integer class values, any fractional values will be rounded in the results. This is explained here:
which states "Arithmetic operations that involve both integers and floating-point numbers always result in an integer data type. MATLAB rounds the result..."
Calling FIX or FLOOR afterwards on some integers does absolutely nothing. And naturally some input values will round up, as your example values show:
col = int32(559) % use the correct INT32 type, not like your test.
col = int32 559
cs = fix((col-1)/4) + 1
cs = int32 141
Lets do it step-by-step:
tmp = col-1
tmp = int32 558
tmp = tmp/4 % note this is already ROUNDed, and the output is also integer type!
tmp = int32 140
tmp = tmp+1
tmp = int32 141
tmp = fix(tmp) % ... so FIX/FLOOR does absolutely nothing.
tmp = int32 141
If you do not want rounding then either:
  • modify your algorithm to work correctly with that integer type, or
  • do not perform arithmetic on integer data type: convert that data to DOUBLE, do your arithmetic, and then FIX/FLOOR as required:
fix((double(col)-1)/4) + 1
ans = 140
This is probably what you expected that intermediate value to be:
(double(col)-1)/4 % not 140
ans = 139.5000
Note that floating point has other issues that you will need to consider.
Even more important: learn to debug. Code does not care what you think it should be doing. It is your task to look at what it really is doing. And the first and most basic step in that, is to look at the data at every step.
  1 Comment
Walter Roberson
Walter Roberson on 10 May 2023
You are right, my guess about the cause turns out to wrong for the circumstances.

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 10 May 2023
I predict that your col was not calculated using pure integers, and instead was calculated by a process that divided by a number and later multiplied by the number. For example if you had (0:0.1:5) * 10 then you will not get back perfect integers:
x = 0:0.1:5;
x(x*10 ~= round(x*10))
ans = 1×10
0.3000 0.6000 0.7000 1.2000 1.4000 2.3000 2.4000 2.6000 2.7000 3.1000





Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!