It has been a while since I used these methods, so I will not go that deeply into the answer. But there are multiple ways to solve a differential equation, not just the ones you find in ODE45 or dsolve. Well, to try to solve one at least. And if the ODE is nonlinear, I would not expect any real hope that you will get an analytical solution.
Here is your ODE:
Well, not too messy at least. there are some common ways you might decide to solve this that do not fall into the standard methods described above, that will convert the problem into a set of (nonlinear) algebraic equations.
Simplest is to discretize the problem, using finite differences. Thus over some domain, we will approximate y at a set of points. Suppose you intend to solve the ODE on the domain [0,a]. Choose some number of points (n) to break the domain into, such that y is now some vector of values at the locations
At each location, we will formulate one nonlinear equation, using a finite difference operator on that third derivative. (Be careful near the ends of the domain, since you may need to tweak that finite difference to be valid.) Create several additional equations to represent the initial/boundary conditions on the ODE. In the end, you want n nonlinear equations in the unknown vector y.
The solution is simple if the ODE is linear, because the result will be a linear system of algebraic equations. But it is only an approximation, as good as the discretization you have chosen. It will NOT yield an analytical solution.
There are other approaches. For example, you might decide to represent y as a spline. Given the third derivative in your problem, I might decide to work with a quintic spline. You should be able to see where this is going, since a spline model will be nicely differentiable. Substitute into the ODE, and the result will again be a system of nonlinear equations in the coefficients of the spline. Again, this cannot possibly result in an analytical solution to your problem.
If I had to guess however, you are thinking about a Galerkin method. Here we will approximate y(x) using some intelligently chosen expansion. Some common choices might be to represent y(x) as a Fourier series, or a sum of Bessel functions, or perhaps a sum of orthogonal polynomials, thus Chebychev, etc. (Were you to go the last route, you might find the chebfun toolbox of great value. Read here, for example. ) So we will write y(x) as a sum of terms a_n*y_n(x), for a set of undetermined coefficients a_n. Form a dot product with each member of the orthogonal family, which results in one nonlinear equation per dot product. Again, solve for the undetermined coefficients in the series expansion. Will this yield an analytical solution? Of course not, unless you can find something nice happens. However good things never seem to happen on these things, at least not to me.
So, all of the above approaches can be made to result in a system of nonlinear algebraic equations to solve. NONE of them will generally yield an analytical solution on a nonlinear problem.
For example, were I to try to solve your ODE
on the domain [0,1], using chebfun. I'll be pretty arbitrary, with y(0)=1, and y'(0)=0, y''(0)=0.
L = chebop(0, 1);
L.op = @(x,u) diff(u,3) + (1-x^2)*u;
L.lbc = [1 0 0];
u = L\0
chebfun column (1 smooth piece)
interval length endpoint values
[ 0, 1] 13 1 0.85
vertical scale = 1
But again, is this an analytical solution? Of course not. It was easy to solve though. Trivial to write using chebfun. I'd give chebfun a try here. (I used the current version found on the FEX, chebfun-chebfun-f5d7a97, for my solution.) While it will not give you the analytical solution you desire, it might give you some intuition about the solution.
On the other hand, essentially NOTHING you do using fsolve will yield an analytical solution anyway.