I am getting increasingly confused with the notation $< , >$ I know the definition of inner-product (simply, dot product) and I also know what a vector is.
Does it have two different meanings that are completely unrelated?
For example we may define some vector $v \in \mathbb{R}^2$ as $<i,j>$ I understand the meaning and context behind this.
But, now take the Gradient Inequality theorem.
$f(x) \geq f(y) + <\Delta f(y), x-y>$ What exactly is the notation saying here?
Is $<\Delta f(y), x-y>$ just a shorthand way of expressing the limit for $\frac{f(y+t(x-y))-f(y))}{t}$ ? Does it ever have anything to do with dot product in this instance?
Does this notation have a completely different meaning depending on the context?
While using left and right brackets is common notation for most introductions to vector algebra/calculus, I've never seen it used in an advanced math textbook. Instead authors like to use $(a,b,c)$ or $a{\bf e}_1+b{\bf e}_2+c{\bf e}_3$ for vectors (the ${\bf e}$'s are the standard basis vectors for $\mathbb{R}^3$--you sometimes see them written as ${\bf i}, {\bf j}, {\bf k}$). The brackets $\langle \cdot,\cdot\rangle$ are indeed reserved for the inner product (which is just the dot product in $\mathbb{R}^n$) in most math past intro courses.
Assuming then that $f: \mathbb{R}_{uv}^2\to\mathbb{R}$, and that $x=(x_1, x_2)$, $y = (y_1, y_2)\in\mathbb{R}^2$, the expression from the gradient inequality would be $$ \langle \nabla f(y), x-y\rangle = \left(\frac{\partial f(y_1, y_2)}{\partial u}, \frac{\partial f(y_1,y_2)}{\partial v}\right)\cdot (x_1-y_1, x_2-y_2) = \frac{\partial f(y_1,y_2)}{\partial u}(x_1-y_1)+\frac{\partial f(y_1,y_2)}{\partial v}(x_2-y_2).$$