If you are attempting to use pass-by-reference to modify the input argument passed into a function, the answer to the question depends on whether the input is a handle object or a value object. The two types of objects are described in the Object-Oriented Programming
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_oop/brfylq3.html and Programming Fundamentals http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/brtm8si.html sections of the documentation. By default, objects (including matrices, arrays, etc. of the built-in data types) are value objects.
Handle objects do exhibit reference behavior when passed as function arguments; value objects do not. When you pass a handle object to a function, MATLAB still copies the value of the argument to the parameter variable in the function (with one bit of subtlety; see below.) However, all copies of a handle object refer to the same underlying object.
If a function modifies a handle object passed as an input argument, the modification affects the object referenced by both the original and copied handles. In this case, the function does not need to return the result to be reassigned.
If instead you are attempting to use pass-by-reference to avoid unnecessary copying of data into the workspace of the function you're calling, you should be aware that MATLAB uses a system commonly called "copy-on-write" to avoid making a copy of the input argument inside the function workspace until or unless you modify the input argument. If you do not modify the input argument, MATLAB will avoid making a copy. For instance, in this code:
function y = functionOfLargeMatrix(x)
y = x(1);
MATLAB will not make a copy of the input in the workspace of functionOfLargeMatrix, as x is not being changed in that function. If on the other hand, you called this function:
function y = functionOfLargeMatrix2(x)
x(2) = 2;
y = x(1);
then x is being modified inside the workspace of functionOfLargeMatrix2, and so a copy must be made.
For more information on this behavior, read this posting http://blogs.mathworks.com/loren/2006/05/10/memory-management-for-functions-and-variables/ on Loren Shure's blog.
[From the MATLAB FAQ of Ancient Times]