- modify your algorithm to work correctly with that integer type, or
- do not perform arithmetic on integer data type: convert that data to DOUBLE, do your arithmetic, and then FIX/FLOOR as required:
Something wrong with "floor" or "fix" functions or it's my code?
6 views (last 30 days)
row and col contain a lot of values of rows and cols. I'm trying to scale down the rows and cols to 1/4 sizes (rs,cs) and ps is the linear index in the scaled down matrix.
rs = fix((row-1)/4) + 1; %scale down of rows
cs = fix((col-1)/4) + 1; %scale down of cols
ps = (cs - 1)*140 + rs; %linear index in 140x140 matrix
It complains that the Index must not exceed 19600. I isolated the problem whenever the col = 559, the cs = 141 instead of 140. If I run the code individually on the command window >> cs = floor((559-1)/4) + 1; or >> cs = fix((559-1)/4) + 1; it gives me the correct value 140.
Anyone sees the problem that I can't see? I appreciate your help.
Stephen23 on 10 May 2023
Edited: Stephen23 on 10 May 2023
"Something wrong with "floor" or "fix" functions"
Nope, in fact those functions do absolutely nothing in your code.
"or it's my code?"
"Anyone sees the problem that I can't see?"
The problem is that your data are pure integers. MATLAB clearly tells us that they are INT32 class:
S = load('variables.mat')
When performing arithmetic on integer class values, any fractional values will be rounded in the results. This is explained here:
which states "Arithmetic operations that involve both integers and floating-point numbers always result in an integer data type. MATLAB rounds the result..."
Calling FIX or FLOOR afterwards on some integers does absolutely nothing. And naturally some input values will round up, as your example values show:
col = int32(559) % use the correct INT32 type, not like your test.
cs = fix((col-1)/4) + 1
Lets do it step-by-step:
tmp = col-1
tmp = tmp/4 % note this is already ROUNDed, and the output is also integer type!
tmp = tmp+1
tmp = fix(tmp) % ... so FIX/FLOOR does absolutely nothing.
If you do not want rounding then either:
fix((double(col)-1)/4) + 1
This is probably what you expected that intermediate value to be:
(double(col)-1)/4 % not 140
Note that floating point has other issues that you will need to consider.
Even more important: learn to debug. Code does not care what you think it should be doing. It is your task to look at what it really is doing. And the first and most basic step in that, is to look at the data at every step.
More Answers (1)
Walter Roberson on 10 May 2023
I predict that your col was not calculated using pure integers, and instead was calculated by a process that divided by a number and later multiplied by the number. For example if you had (0:0.1:5) * 10 then you will not get back perfect integers:
x = 0:0.1:5;
x(x*10 ~= round(x*10))