I am given the following non-linear second order ode:
$$x''=mg\sin(\theta)+k(L-\sqrt{x^2+h^2})\left(\frac{x}{\sqrt{x^2+h^2}}\right)-bx'$$
where $x$ is a function of $t$ or time. I wrote this equation as the following system:
$$x_1'=x_2$$
$$x_2'=mg\sin(\theta)+k(L-\sqrt{x_1^2+h^2})\left(\frac{x_1}{\sqrt{x_1^2+h^2}}\right)-bx_2$$ where $x_1=x$ and $x_2=x'.$ The equilibrium points of this system satisfy the equation $x_1'=x_2'=0$ which means that $$x_2=0$$ and $$mg\sin(\theta)=kx_1\left(1-\frac{L}{\sqrt{x_1^2+h^2}}\right).$$ If we denote the solutions to the latter equation as $x_1^*$ then $(x_1^*,0)$ are fixed points of our system. Then we need to check their stability and for that we perturb our system. In other words we do the following: $$u=x_1-x_2^*$$ $$v=x_2-0.$$ Then as $u'=x_1'=f(x_1^*+u,x_2)$ by the Taylor series expansion centred at the fixed point we obtain $$u'=u\frac{\partial f}{\partial x_1}+v\frac{\partial f}{\partial x_2}+\text{higher order terms},$$ where $x'=f(x_1,x_2)=x_2.$ So we have that $$u'=v.$$ And similarly $$v'=u\underbrace{\left(kL\left(\frac{1}{\sqrt{(x_1^2+h^2)}}-\frac{x_1^2}{(x_1^2+h^2)^{3/2}}\right)-k\right)}_{f(x_1)}-bv+\text{higher order terms}.$$
Hence we can get the following linearised system: $$\begin{bmatrix} u' \\ v' \end{bmatrix}=\begin{bmatrix} 0 & 1\\ f(x_1) & -b \end{bmatrix}\begin{bmatrix} u \\ v \end{bmatrix}.$$
Have I made any mistake in my analysis till here because I am getting the wrong conclusions after this? Perhaps someone can give some suggestions?
First of all, with your analysis you have managed to transform your system's equilibrium to the origin, that is, $(0,0)$ to be the solution of the following system:
\begin{equation}u' = 0\\ v'=0\end{equation}
Now, in order to compute the linearized system that accords to your own, you have to compute the Taylor expansion locally and close to the fixed point, that is, the components:
\begin{equation} \frac{\partial}{\partial x_1}f_1 , \frac{\partial}{\partial x_2}f_2\end{equation}
should be computed at $(x_1,x_2)=(x_1^*,x_2^*)$. The linearization should lead to a linear system:
\begin{equation} u' = \alpha u + \beta v \\ v' = \gamma u + \delta v \end{equation}
which in vector form can be written as:
\begin{equation} \begin{bmatrix}{u'\\v'}\end{bmatrix}=\begin{bmatrix}\alpha & \beta \\ \gamma & \delta \end{bmatrix}\begin{bmatrix}u \\ v\end{bmatrix}\end{equation}
Notice that the associated matrix should be linear. In your study, you clearly have a nonlinear function $f(x_1)$ instead of a number. So, to answer more properly, I will give you a quick notion of how to linearize your system properly. Consider an $n-$dimensional vector field:
\begin{equation}X'=G(X) \end{equation}
where $X=(x_1,x_2,...,x_n)$ is an $n-$ dimensional vector. $G=(g_1(X),g_2(X),...,g_n(X))$ is the vector of the right hand side functions. In your case, n=2 and
\begin{equation} X = \begin{bmatrix} u \\ v \end{bmatrix} , G = \begin{bmatrix}u' \\ v' \end{bmatrix}\end{equation}
To compute the associated linearized system at a fixed point (say, the origin $(0,0)$), you have to evaluate the following matrix:
\begin{equation} J=\begin{bmatrix}\frac{\partial}{\partial x_1} g_1 & \frac{\partial}{\partial x_2}g_1 & \cdots & \frac{\partial}{\partial x_n}g_1 \\\frac{\partial}{\partial x_1}g_2 & \frac{\partial}{\partial x_2}g_2 & \cdots & \frac{\partial}{\partial x_n}g_n \\\vdots & \vdots & \ddots & \vdots \\\frac{\partial}{\partial x_1}g_n & \frac{\partial}{\partial x_2}g_n &\cdots & \frac{\partial}{\partial x_n}g_n \end{bmatrix} \end{equation}
The above matrix is called the Jacobian matrix. Remember that every derivative is computed at the fixed point i.e. at the origin where $(u,v)=(0,0)$. The Jacobian matrix is the coefficient matrix of the linearized system:
\begin{equation} X'=JX \end{equation}
and for the 2-dimensional case:
\begin{equation} \begin{bmatrix} u' \\ v' \end{bmatrix} = J \begin{bmatrix} u \\ v \end{bmatrix} \end{equation}
I hope I have helped you somehow. A good introduction to linear algebra and dynamical systems is the following aticle and the references therein: Scholarpedia-Equilibrium