I'm trying to use gradient descent to minimize an objective function. Part of the function is
$$F (x) = \|A - xB\|^2$$
where $A$ and $B$ are matrices, $x$ is a scalar and $\| \cdot \|$ denotes the Frobenius norm.
Using gradient descent, we have:
$$\delta F / \delta x = -2B(A-xB)$$
Then we update $x$:
$$x = x - \delta F / \delta x = x +2B(A-xB)$$
In this equation, since $A$ and $B$ are matrices, so $2B(A-xB)$ is a matrix and can't be subtracted from $x$ which is a scalar. Can someone explain to me how I can update $x$ here?
\begin{align} F (x) &= \|A - xB\|^2_F = \sum_{i,j} (A_{ij} -x B_{ij})^2 \\ \partial_x F (x) &= \partial_x \sum_{i,j} (A_{ij} -x B_{ij})^2 \\ &= \sum_{i,j} 2(A_{ij} -x B_{ij})\partial_x (A_{ij} -x B_{ij}) \\ &= -2 \sum_{i,j} (A_{ij} -x B_{ij})(- B_{ij}) \\ &= -2\left[ \sum_{i,j}A_{ij}B_{ij} - xB_{ij}^2 \right] \\ &= -2\left[ S(A\odot B) - x ||B||_F^2 \right] \end{align} where $\odot$ is the Hadamard product and $S(M) = \sum_{i,j} M_{i,j}$.