What does it mean for the derivative of a dot product to be a vector?

299 Views Asked by At

For S = $a^tb$ where $a$, $b$ are vectors in an n dimensional space

$dS/db$ = $a$ (mostly im missing something really obvious here)

What does it even mean for the derivative of a scalar function with respect to a vector to be a vector? Should it not be a scalar (which is the directional derivative of $S$ in the direction of $b$)?

1

There are 1 best solutions below

5
On BEST ANSWER

$$S(b)=a^Tb = \sum_{i=1}^n a_i b_i$$

what happens if I perturb $b_1$? how will the value change?

$$\frac{\partial{S}}{\partial b_1}=a_1$$

The value of $S$ will change by $a_1$ for every unit change in $b_1$.

What about $b_2$?

$$\frac{\partial{S}}{\partial b_2}=a_2$$

The value of $S$ will change by $a_2$ for every unit change in $b_2$.

$\frac{d{S}}{d b}$ effectively encode all the information in each component in a single vector.

Edit:

I believe what are you looking for is directional derivative.

what is the rate of change of $S$ when I move along a non-zero direction $u$?

$$\nabla_u S = \nabla S. \frac{u}{|u|} = \lim_{h \rightarrow 0} \frac{S(b+uh)-S(b)}{h}$$

Here $\nabla S = \frac{dS}{db}$.