MOTIVATION:
If you apply Newton's basic rules you got this
$$m\ddot x_1=-kx_1+k(x_2-x_1)\\ m \ddot x_2 = -k(x_2-x_1)+k(-x_2)$$
Writing this in linear system form we got:
$$\ddot X=AX\\ \equiv \\ \ddot X= \dfrac k{m}\begin{bmatrix} -2 & 1 \\ 1 & -2 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$$
My Problem: In school I only learned how to solve first order linear system of differential equation. In classical mechanics courses, my ignorant lecturer solves this to converting to first order linear system but no one can understand how he does it. Before posting it I found some sources solving it by diagonalization or some other method but how and why? What is the logic, in first place why cannot I solve 2nd order linear systems? How I interchange it (what is the MATHEMATICAL reasoning)?

The usual way to a first order system, mostly for numerical integration, is to set $V=\dot X$.
You can also go the other way, the characteristic polynomial of $\pmatrix{-2&1\\1&-2}$ is $\lambda^2+4λ+3=0$ so that solutions also satisfy $[(\frac mk D^2)^2+4(\frac mk D^2)+3I]X=0$ or $$ m^2 X^{(4)}+4mk\ddot X+3k^2X=0, $$ which can be solved for the first componenent separately as a scalar ODE, linear with constant coefficients, and after that one can use the first line of the given equation to get $x_2=2x_1+\frac mk\ddot x_1$.
If you know the eigenvectors, you know what linear combinations to take to decouple the system. Here you get for instance in the sum of the equations
$$ \ddot x_1+\ddot x_2 = -\frac{k}m(x_1+x_2) $$ and in the difference $$ \ddot x_1-\ddot x_2 = -3\frac{k}m(x_1-x_2) $$ which are both harmonic oscillations for the combined terms that can be easily solved and then recombined into solutions for the components.