Learning some weight vector $\mathbf{w}$ is known to be an instable point in a dynamic system, since the growth of $\mathbf{w}$ is unbounded. The learning rule for a given input vector $\mathbf{x}$ and some learning rate with $0 < \eta << 1$ is given as
$$ \mathbf{w}(t+1) = \mathbf{w} + \eta\mathbf{w}^T\mathbf{x}\mathbf{x}$$
To tackle the problem of the unstable dynamic $\mathbf{w}$ is normalized, such that we have
$$ \mathbf{w}(t+1) = \frac{\mathbf{w} + \eta\mathbf{w}^T\mathbf{x}\mathbf{x}}{||\mathbf{w} + \eta\mathbf{w}^T\mathbf{x}\mathbf{x}||}$$
From here Oja's learning rule can be derived using power series expansion. I found some explanations on p=2 in "Haykin - Neural Networks and Learning Machines", but I want to understand the general case. I have a problem seeing on how to apply the power series expansion on this equation. Wikipedia has the closest version of an explanation on the general case. It is given as
$$ w_i (n+1) ~ = ~ \frac{w_i(n)}{\left( \sum_j w_j^p(n) \right)^{1/p}} ~ + ~ \eta \left( \frac{y x_i}{\left(\sum_j w_j^p(n) \right)^{1/p}} - \frac{w_i(n) \sum_j y x_j w_j^{p-1}(n)}{\left(\sum_j w_j^p(n) \right)^{(1 + 1/p)}} \right) ~ + ~ O(\eta^2)$$
But how are the parameters of the power series chosen, such that we get this equation? Is there also a way to give the power series expansion on the norm notation itself?
This looks like a series expansion in $\eta$ around the point $\eta = 0$. So \begin{align*} w_i(n+1) = \sum_{k = 0}^\infty \frac{\eta^k}{k!}\dfrac{d}{d\eta}\big|_{\eta = 0}w_i(n+1) \end{align*} and then plugging the formula in on the right side you get the first terms as: \begin{align*} w_i(n+1) = \frac{w_i(n)+\eta (w^Tx)x_i}{|| w+\eta (w^Tx)x ||_p}\big|_{\eta = 0} +\eta\cdot \dfrac{d}{d\eta}\big|_{\eta = 0}\frac{w_i(n)+\eta (w^Tx)x_i}{|| w+\eta (w^Tx)x ||_p} + O(\eta^2) \end{align*} Now you just have to compute the derivative and then take the limit $\eta \rightarrow 0$