Is there a way to find the minimum eigenvalue of a matrix analytically? (I know that the usual way of finding the eigenvalues is the characteristic polynomial).
One definition of the minimum eigenvalue is
$$ \min_{\,\,\,x\\x\neq 0} \frac{\|A x\|^2}{\|x\|^2} $$
For the simple matrix [ 0 1 ; -2 -3 ] I tried to find the smallest eigenvalue analytically by naming the x-vector components $x_1$ and $x_2$ and having $D = ((-2x_1 -3 x_2)^2 + x_2^2)/(x_1^2+x_2^2)$. From this I obtain the gradient $\frac{\partial D}{\partial x_1}$, $\frac{\partial D}{\partial x_2}$ and set it to zero solve for $x_1$ and $x_2$.
Surely enough, this system of equations does not have a solution.
Your "definition" of minimum eigenvalue has several problems. For one thing, it would imply that every matrix has a real and positive eigenvalue, which is not true in general. Also, dont' forget a simple sanity check: your matrix property should apply for a $1\times 1$ matrix... and it clearly doesn't.
What you actually mean, I guess, is : if $M$ is hermitian and positive definite (which implies that its eigenvalues are real and positive), then letting $\lambda_1$ be its smallest eigenvalue, we have
$$\lambda_1 = \min_{x \ne 0} \frac{x^t M x }{x^t x} $$
Further, if we decompose $M=A^t A$ the above gives
$$\lambda_1 = \min_{x \ne 0} \frac{x^t A^t A x }{x^t x} =\min_{\,\,\,x\\x\neq 0} \frac{\|A x\|^2}{\|x\|^2}$$
But, bear in mind, here $\lambda_1$ is an eigenvalue of $M$, not of $A$.
To answer the question in the title: there cannot be a general analytical procedure for computing the smallest eingenvalue of a matrix, because the problem maps to finding the smallest root of a polinomial, and for degrees five and above there's no analytical procedure (in the usual sense of the expression) for doing that (Abel-Ruffini theorem)
On the other side, if the matrix is $2\times 2$, then it's trivial. But you would't use the Rayleigh quotient for that. This is only useful for computing numerically the smallest (or biggest) eigenvalue, by an iterative method.