I'm currently reading following paper: https://www.ri.cmu.edu/pub_files/pub3/baker_simon_2001_2/baker_simon_2001_2.pdf
Here I struggle to follow the Taylor expansion from equation 2 to equation 3. I mean I know in general how Taylor expansion works, but in this case it is not clear to me.
Starting equation:
$$ \sum_{x} [ \mathit{I}(\mathbf{W}(\mathbf{x};\mathbf{p} + \Delta\mathbf{p})) - \mathit{T}(\mathbf{x})]^2 $$
Taylor expansion
$$ \sum_{x} [ \mathit{I}(\mathbf{W}(\mathbf{x};\mathbf{p} )) + \nabla\mathit{I}\frac{\partial\mathbf{W}}{\partial\mathbf{p}}\Delta\mathbf{p} - \mathit{T}(\mathbf{x})]^2 $$
Maybe somebody of you has more experience and can elaborate little more on this?
You just need to apply the expansion twice. First for ${\bf W}$ up to first order in $\Delta {\bf p}$, we get
$$ {\bf W}({\bf x}; {\bf p} + \Delta {\bf p}) \approx {\bf W}({\bf x}; {\bf p}) + \frac{\partial {\bf W}}{\partial {\bf p}}\Delta {\bf p} \equiv {\bf W} + \Delta{\bf W}\tag{1} $$
Now for $I$ up to first order in $\Delta {\bf W}$,
$$ I({\bf W} + \Delta{\bf W}) \approx I({\bf W}) + \nabla I \Delta{\bf W} = I({\bf W}) + \nabla I \frac{\partial {\bf W}}{\partial {\bf p}}\Delta {\bf p} \tag{2} $$
Putting everything together
$$ I({\bf W}({\bf x}; {\bf p} + \Delta {\bf p})) \approx I({\bf W}({\bf x}; {\bf p})) + \nabla I \frac{\partial {\bf W}}{\partial {\bf p}}\Delta {\bf p} $$