Upper bound for Telescoping sum in gradient descent

119 Views Asked by At

I am studying a chapter in gradient descent . At some point we reach the sum in the left of the enequality and the writer says it's telescopic so this enequality holds:
$\sum_{t=1}^T \Big( ||x_t - x^*||^2 - ||x_{t+1}-x^*||^2) \leq ||x_1-x^*||^2 $

It's not so obvious to me though. I tried to compute the sum and I think it is :
$\sum_{t=1}^T \Big( ||x_t - x^*||^2 - ||x_{t+1}-x^*||^2) =x_1^2 -x_T^2 -2x^*x_1 + 2x^*x_T $ It does not look like the norm above .. actually I need somehow to get rid of $x_T$ but I have no clue how..
$x_t$'s (belong to a convex interval)

1

There are 1 best solutions below

0
On BEST ANSWER

You always have$$\sum_{t=1}^T(a_t-a_{t+1})=a_1-a_{T+1}.$$Therefore,\begin{align}\sum_{t=1}^T\left(\|x_t-x^*\|-\|x_{t+1}-x^*\|\right)&=\|x_1-x^*\|-\|x_{T+1}-x^*\|\\&\leqslant\|x_1-x^*\|.\end{align}