dotReference with matrix indexing and "end" keyword

Somewhat related to this https://www.mathworks.com/matlabcentral/answers/2102526-when-would-dotlistlength-as-part-of-matlab-mixin-indexing-redefinesdot-return-something-other-than-1 but I figured the question is different enough to split it into a separate question.

In the following example, I'm noticing something strange when addressing a matrix with "end" keyword:

classdef ScalarStructClass2 < matlab.mixin.indexing.RedefinesDot & ...
        matlab.mixin.Scalar
    
    properties (Dependent, SetAccess=private)
        FieldNames
    end
    properties (Access=private)
        AddedFields struct
    end
    methods
        function out = get.FieldNames(obj)
            out = string(fieldnames(obj.AddedFields));
        end
    end
    methods (Access=public)
        function obj = ScalarStructClass2(fieldName,fieldValue)
            if nargin == 1
                obj.AddedFields = fieldName;
                return;
            end
            obj.AddedFields = struct(fieldName,fieldValue);
        end
        function out = getAddedFields(obj)
            out = obj.AddedFields;
        end
        function [a, b] = getSomething(obj)
            a=1;
            b=2;
        end
    end
    methods (Access=protected)
        function varargout = dotReference(obj,indexOp)
            fprintf(1, "At dotReference\n");
            disp(indexOp)
            [varargout{1:nargout}] = obj.AddedFields.(indexOp);
        end
        function obj = dotAssign(obj,indexOp,varargin)
            [obj.AddedFields.(indexOp)] = varargin{:};
        end
        
        function n = dotListLength(obj,indexOp,indexContext)
            n = listLength(obj.AddedFields,indexOp,indexContext);
            fprintf(1, "Num Fields: %i\n", n);
        end
    end
end

myStructClass = ScalarStructClass2("Field1",75)
myStructClass.testField = randn(6,6); 

"testField" is a 6 by 6 matrix. I access it with

myStructClass.testField(2:4,3:end)

This calls "dotReference" twice with different indexOp passed into it:

At dotReference
IndexingOperation with properties:

Type: Dot
Name: "testField"

At dotReference
1×2 IndexingOperation array with properties:
Type
Name       (when Type is Dot)
Indices    (when Type is Paren)

It seems the first dotReference call has indexOp without the parenthesis while the 2nd call has the Paren and the Indices are resolved:

K>> indexOp(2)

ans = 

IndexingOperation with properties:

Type: Paren
Indices: {[2 3 4]  [3 4 5 6]}

This seems to be the behavior is index with "end" is used. If "end" is not used in the index, dotReference is only called once. I suspect that this is done to resolve what "end" is in terms of numbers.

The problem for my real use case is that I'm overloading this operation to fetch the values from a server off of my machine. So querying the server can be slow, especially for large values.

However, the size of the variable is stored in Matlab. Ideally, I would like to have one call into dotReference with the "end" keyword if needed, and I can calculate the indices myself. Is something like this possible?

Answer 1

[Kyle]

Hi Cole,

This is a good question. As you noted, end will create a call to dotReference because of the syntax present in your expression, myStructClass.testField(2:4,3:end). This doc page provides some details about the way the end function takes myStructClass.testField as its first input. You can observe dotReference being called a second time because MATLAB must evaluate myStructClass.testField to pass the result to end .

You already identified that omitting end allows you to work around your performance bottleneck. Another solution that might work better in your case is to split the dot and paren indexing into different statements, like so:

myTestField = myStructClass.testField;
myTestField(2:4,3:end)

With this approach, the end function will be passed myTestField instead of myStructClass.testField, so dotReference will not need to be called. Another way of thinking of this solution is that you are effectively "caching" the result of dotReference in a workspace variable named myTestField so that you can reuse it without calculating it a second time.

[Cole]

Thanks for the clarification Kyle.

In my use case, I'm writing a framework where other users would write code on top of this. So having them do something like what you proposed isn't ideal, but I will make a note of it.

Ideally, it would be great if there's a way where I can somehow do this "caching" operation within the context of dotReference. Currently, there's no way for me, as the dotReference implementer, to know if the call is with or without 'end'. Because if there is, then I can much more intelligently handle this case since I know the size of the matrix.

[Kyle]

Given your use case, I can see why that would be more convenient.

Like most languages, when a MATLAB function is called, all inputs must be resolved before being passed in, so dotReference will always see the result of evaluating end. You might not be able to detect if end is one of the inputs, but you might be able to achieve the caching some other way.

I have seen some users who encounter poor get method performance solve it by implementing a "property cache" for their get methods. They do this by storing the cached property in some other property on the object and updating this property every time its return value would be modified. The get method doesn't calculate the property every time it is accessed, but instead stores the new value every time it is modified. This approach is better for cases where get is called much more frequently than set (or is much more expensive than set), because the trade-off is more time spent when changing this cache any time its value would change.

In your case, you may be able to do something similar. I do not know what the performance profile of your code normally looks like, but you could write logic that caches myStructClass.testField itself. Is it possible to store the value locally any time the "answer" to myStructClass.testField would change? Theoretically, if dotAssign has not been called on your object with the field named "testField", I would expect that myStructClass.testField should be the same as the last time it was queried. Then, rather than forwarding your indexing to the server to calculate statements, you could instead forward the indexing to your cached value of myStructClass.testField.

It might be that the server state is what changes and you do not have a way to know if the results are still valid. Or maybe the data is large enough that caching isn't possible. In those cases, avoiding the performance cost of using end will be harder to avoid.

[Cole]

Good idea on the cache.

In my use case, yeah, the server could potentially change the value behind my back. But I don't anticipate it to be TOO rapid. So I suppose I can have an internal counter that keeps track of when the last time dotReference was called and if it's below a certain threshold, I will return the cached value. Otherwise, I'll retrieve it from the server, update the cache, and update the last access time.

Thanks for giving me that time!