For scalar-valued problems, the function y = fun(x) must accept a vector argument, x, and return a vector result, y. This generally means that fun must use array operators instead of matrix operators. For example, use .* (times) rather than * (mtimes). If you set the 'ArrayValued' option to true, then fun must accept a scalar and return an array of fixed size.
You have not set the option 'ArrayValued', so your input value t will be a vector of arbitrary length.
With some research into the code for integral() I see that the parameter it passes into the function, when ArrayValued is not set, is a row vector and you effectively have no control over its length.
Your "x" is a row vector, of length you pre-set.
In your function you have sin(t*x) and cos(t*x). You are attempting to do a matrix multiplication of two row vectors, which is going to fail automatically unless one of them is a scalar. But they aren't even the same length of row vectors so you cannot transpose one so that you are multiplying row vector times column vector giving a scalar as a result. You could transpose one and have column vector times row vector: in that case the inner dimensions would both be 1 and you would get an array result that was the length() of one of the vectors rows and length() of the others in columns.
You might at this point have decided that you are trying to produce array valued result and want the ArrayValued flag to be set. But I don't know as other parts of your function are unclear, so I will continue the analysis assuming that the correction is going to be to do the transpose so you get out length(x) x length(t) or else length(t) x length(x) as an intermediate array.
Once you have taken cos(t*x) or sin(t*x) you transpose the result. So now you are working with length(t) x length(x) or else length(x) x length(t).
You have ones(1,n) * that result. In order for that to have a chance of working, the inner dimensions must agree, so the right side must be n x something. Since we don't control length(t) we have to make sure that length(x) is the first dimension, and that n = length(x). Which might be the case, but if so it would have been nice if you had said so in order to save a bunch of reasoning.
Assuming then that n = length(x), since the length(x) has to occur first after the transpose, it must have started as length(t) x length(x) for the array. To get that, it has to be the t row vector that we transpose and have first, so instead of cos(t*x) or sin(t*x) we need to instead of cos(t'*x) and sin(t'*x). With that done, the result of the ones(1,n)*cos(t'*x)' would be 1 x length(t). And that is the size of the output that we need to produce, so that would be okay.
The most likely fix would thus be to transpose the t row vector in the places it is used, and to add the condition that n = length(x)
