Let the following recurrence relation with non-constant coefficients be defined:
$a_0 = 1$
$a_1 = x$
$a_n = \frac{x}{n} \left( a_{n-1} + a_{n-2} \right)$
My attempt:
Let's define our target function as: $y = \sum\limits_{n=0}^{\infty} a_n x^n$
$n a_n = x a_{n-1} + x a_{n-2}$
$\sum\limits_{n=2}^{\infty} n a_n x^{n-1} = x \sum\limits_{n=2}^{\infty} a_{n-1} x^{n-1} + x^2 \sum\limits_{n=2}^{\infty} a_{n-2} x^{n-2}$
$\sum\limits_{n=1}^{\infty} \left( n a_n x^{n-1} \right) - x = x \sum\limits_{n=1}^{\infty} \left( a_{n-1} x^{n-1} \right) - x + x^2 \sum\limits_{n=2}^{\infty} a_{n-2} x^{n-2}$
$y' - x = xy - x + x^2y$
$y' = y \left( x^2 + x \right)$
$\int y^{-1} dy = \int \left( x^2 + x \right) dx$
$\ln{|y|} = \frac{x^3}{3} + \frac{x^2}{2} + C$
With $y(0) = 0$:
$y = e^{\frac{2x+3}{6}x^2}$
But, if I am comparing manually the plots, which I get for the recurrence $a_n(x)$ and the solved function $y(x)$, they become not the same for raising $n$. Even at $|x| < 1$.
Question a) What is wrong with my solution for $|x| < 1$ ?
Question b) Is there an other way to solve it? Especially for for $|x| \geq 1$ ?
You are mixing up $x$ (the parameter in your recurrence) with the formal variable used to write a generating function. That does work here, but not as you attempt to do it.
Define $A(z) = \sum_{n \ge 1} a_n z^n$, Write the recurrence as $(n +2) a_{n + 2} = x a_{n + 1} + x a_n$. Multiply by $z^n$, sum over $n \ge 0$ and decipher the resulting sums:
$\begin{align*} \sum_{n \ge 0} (n + 2) a_{n + 2} z^n &= x \sum_{n \ge 0} a_{n + 1} z^n + x \sum_{n \ge 0} a_n z^n \\ &= x \frac{A(z) - a_0}{z} + x A(z) \\ \end{align*}$
For the left hand side, it looks similar to:
$\begin{align*} z A'(z) &= \sum_{n \ge 0} n a_n z^n \\ \frac{z A'(z) - 0 \cdot a_0 - 1 \cdot a_1 z} {z^2} &= \sum_{n \ge 0} (n + 2) a_{n + 2} z^n \end{align*}$
Putting everything together you have a differential equation for $A(z)$ (linear, first order; thus solvable). For initial value you have $A(0) = a_0$. Having $A(z)$, extracting coefficients gives the $a_n$.