Taking derivative of objective function

Question

Given $x_t$ and $c_n$, the objective function is defined as follows

$$J (\theta) := \sum_{t=1}^L \Big( x_t - \sum_{n=1}^N c_n \, \underbrace{\exp \left( -\left(\frac{t-n}{\theta}\right)^2 \right)}_{=: \varphi_{tn} (\theta)}\Big)^2 = \sum_{t=1}^L \Big( x_t - \sum_{n=1}^N c_n \, \varphi_{tn} (\theta) \Big)^2$$

Can somebody help me solve for $\theta$? I am a novice at solving optimization problems, so could somebody please tell me how do I get started to solve this problem using method such as Quasi Newton?

First few steps (edit):

\begin{align} \begin{split}\label{eq:2} \frac{dJ}{d\theta}{}=& 0\\ {\sum_{t}\frac{d}{d\theta}\left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) ^{2}}=& 0 \\ {\sum_{t}-2 \left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) \left( \sum_{n=0}^{N-1} c_{n} \frac{d}{d \theta} e^{-\left(\frac{t-n}{\theta}\right)^2}\right)}=& 0 \\ {\sum_{t}-4 \left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) \left( \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \frac{(t-n)^2}{\theta^3}\right)}=& 0 \\ \end{split}\\ \end{align}

2nd edit: Let $\Phi (\theta)$ be the $m \times n$ matrix whose $(i,j)$-th entry is $\varphi_{ij} (\theta)$. Hence, cost function $J$ can be written more succinctly as follows

$$J (\theta) := \left\| \Phi (\theta) \, \mathrm c - \mathrm x \right\|_2^2 = \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)^\top \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)$$

whose first derivative is

$$J ' (\theta) = 2 \, \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)^\top \Phi ' (\theta) \, \mathrm c$$ $J(\theta)$ over the range of $\theta$ is shown in the plot below:

P.S Matlab code to generate $J(\theta)$

L = 144; % Length of the signal

t = 0:L-1;
a = 1; % time shift factor
n = 0:L/a-1; %time shift index


x = ((-(pi*(t-20).^2)./16.^2))';

theta = 4:4:256;  % differnt variance in samples
D = zeros(L,length(n)); % ,matrix containing time-shifted gaussian

J = zeros(length(theta),1);
for i =1:length(theta)

     for t=1:length(n)
         D(:,t) = TimeShiftedGaussianMatrix(L,a,theta(i),n(t));       
     end

  c = pinv(D)*x;


  J(i) = (norm(x-D*c))^2; 
end

plot(theta,J); hold on

xlabel('\theta') ylabel('||x-\Phi(\theta)c||^2')


function atom = TimeShiftedGaussianMatrix(Lt,a,theta,n)

atom = zeros(Lt,1);
t = 0:Lt-1;
for i = 1:length(t)

 atom(i,1) = exp(-(pi*(t(i)-n*a).^2)./theta.^2);
end

end

Answer 1

Applying chain rule:

\begin{align} \begin{split}\label{eq:1} \frac{dJ}{d\theta}{}=& \frac{d}{d\theta}\left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) ^{2}\\ {}=&-2 \left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) \left( \sum_{n=0}^{N-1} c_{n} \frac{d}{d \theta} e^{-\left(\frac{t-n}{\theta}\right)^2}\right) \\ {}=&-4 \left( x(t) - \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \right) \left( \sum_{n=0}^{N-1} c_{n} e^{-\left(\frac{t-n}{\theta}\right)^2} \frac{(t-n)^2}{\theta^3}\right) \end{split}\\ \end{align}

Edit: I am supposing that all involved variables are scalars

Answer 2

$$J (\theta) := \sum_{t=1}^m \Big( x_t - \sum_{k=1}^n c_k \, \underbrace{\exp \left( -\left(\frac{t-k}{\theta}\right)^2 \right)}_{=: \varphi_{tk} (\theta)}\Big)^2 = \sum_{t=1}^m \Big( x_t - \sum_{k=1}^n c_k \, \varphi_{tk} (\theta) \Big)^2$$

Let $\Phi (\theta)$ be the $m \times n$ matrix whose $(i,j)$-th entry is $\varphi_{ij} (\theta)$. Hence, cost function $J$ can be written more succinctly as follows

$$J (\theta) := \left\| \Phi (\theta) \, \mathrm c - \mathrm x \right\|_2^2 = \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)^\top \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)$$

whose first derivative is

$$J ' (\theta) = 2 \, \left( \Phi (\theta) \, \mathrm c - \mathrm x \right)^\top \Phi ' (\theta) \, \mathrm c$$

One can then attempt to solve $J ' (\theta) = 0$ using, say, Newton-Raphson.