I'm trying to approximate a image with a sum of radial basis functions $\phi(x_i, \theta_i)$ where each $x_i\in R²$ and it is the coordinate of every pixel of the original image. $\theta_i$ is the center of the function.
Then I have to minimize this function $$\dfrac{1}{2}\Vert f - \Phi_\theta\alpha\Vert²$$ with respect to $\theta$ and $\alpha$, where $\Phi_\theta$ is the matrix of radial basis functions and $\alpha$ is the vector that makes all the linear combinations of functions.
This process I get that is in two steps. The first one is minimize $\alpha$ by least squares fixing $\theta$, and the second one is to minimize $\theta$ using gradient descent with the $\alpha$ vector minimized.
To the first step I was able to made it, but I am having problems with gradient descent because I can´t compute the gradient of the objective function with respect to $\theta$. It's important to see that, the gradient must to have the same dimension of $\theta$, the last one is $Jx2$ where $J$ is the number of radial functions that I want to use.
My attempt:
$$\dfrac{1}{2}\Vert f - \Phi_\theta\alpha\Vert² = \dfrac{1}{2}(f - \Phi_\theta\alpha)^T (f - \Phi_\theta\alpha) $$ $$= (f - \Phi_\theta\alpha)^T\dfrac{\partial }{\partial \Phi_\theta}(f - \Phi_\theta\alpha) \dfrac{\partial \Phi_\theta}{\partial \theta}$$ but I don´t know how to continue.