This question overlaps with physics and statistics, but since I'm not getting answers at those SE forums, I will post it here since my difficulties are after all mathematical in nature.
All of the questions posed here are related to this Physics SE post that proves that the propagation of uncertainties is quadratic.
Can you provide further clarification on why $\sum_{i=1}^nb_{ii}E(x_i^2)$ appears in the second equation? Chris here says: To second order, you have to account for the expectations of the squares of the inputs not matching the squares of the expectations. Still, I'm having trouble understanding this. Also in the fourth equation two terms $2f_0\sum_{i=1}^nb_{ii}E(x_i^2)$ and $\sum_{i=1}^na_i^2E(x_i^2)$ appear and I have no idea from where they come from.
From where does the $2$ on the third term of the second equation come from, and why is the second sum in the same term starting from $j=i+1$ when in the first equation it starts simply from $j=1$. Also the same thing appears in equation 4, where the 2 on the third term becomes 4, and again $j=1$ becomes $j=j+1$, the same thing happens with the fourth term.
- Shouldn't the squared expectation of $y$ (third equation) include more terms? If we set $f_0=a$; $\sum_{i=1}^na_i\hat{x_i}=b$; $2\sum_{i=1}^n\sum_{j=i+1}^nb_{ij}\hat{x_i}\hat{x_j}=c$; $\sum_{i=1}^nb_{ii}E(x_i^2)=d$ then $(a+b+c+d)^2=a² + b² + c² + d² + 2ab + 2cd + 2ac + 2bc + 2ad + 2bd $ however I don't see terms $4c^2$,$4bc$,$2bd$,$d^2$ and $4cd$ in the third equation.
- How do we go from the fifth to the sixth equation?
1/2: they are just taking expectations of every term, using linearity of expectation. They then separate out the terms where $i=j$ from the terms where $i \neq j$ (because this will be helpful to compute variances). They then, instead of writing $\sum_{i=1}^n \sum_{j=1,j \neq i}^n$, decide to write $2 \sum_{i=1}^n \sum_{j=i+1}^n$; it is the same because the $(i,j)$ term and $(j,i)$ term are identical in each case.
3: Though I don't want to work out all of the algebra, my guess would be that the terms that are not explicitly included wound up in the $O(x^3)$ term. As an example, if $a=O(x^2),b=O(x),c=O(1)$ then $(a+b+c+O(x^3))^2=a^2+b^2+c^2+2ab+2bc+2ac=b^2+c^2+2bc+2ac+O(x^3)$. Note that the $O(x^3)$ terms are not the same; some of the products involving just $a,b,c$, such as $a^2$, were rolled into the new $O(x^3)$.
4: As best I can tell it is just a change of notation, and an omission of the explicit error term. That is, the variance is exactly the difference in expectations in the sum, and the first derivative is exactly the $a_i$. Then they dropped the error term so $=$ had to become $\approx$.