I'm reading a book, and on its numerical optimization chapter, more precisely in the BFGS algorithm section, the authors state that it's a "rank-two update to the matrix". I've searched for the definition, but I only found the definition for the rank-one update here.
So my question is what is a rank-2 update, and why is the BFGS an example of such an update.
Thanks in advance.
It is two independent rank-one updates performed simultaneously, or if you want, the addition of a rank-2 matrix. This happens because you are updating a symmetric matrix in a symmetric way with the dyadic product $ab^T$ and its transpose $ba^T$, which can also be expressed as some difference of the symmetric updates $(a+b)(a+b)^T$ and $(a-b)(a-b)^T$.
For the secant condition one wants to enforce $B_+s=y$. With a rank-1 update that would give $(B+ab^T)s=y$ giving $a=y-Bs$ and with some other geometric constraints $b=\alpha s$. This is not very symmetric in most cases.
So one tries $B_+=B+ab^T+ba^T$ with $y=B_+s=Bs+a(b^Ts)+b(a^Ts)$. Then again for some kind of geometric minimality one gets that $a$ and $b$ have to be linear combinations of $y$ and $Bs$ which again should be sufficient to determine them.