I have this problem, where the function is to derived with respect to $\theta$
$$f(\theta) = \frac{1}{2} (y-N(x, \theta))^2 $$
I know that I am supposed to use the chain rule here, but I am still a little confused. How can $h(x)' = g(x)*g(x)'$ be of any use here. Any tips are appreciated.
The chain rule is used if there is a concatenation of functions that you want to find the derivative of. In this case, you have the function
$\theta \mapsto y-N(x,\theta)$
first, and that you concatenate with the function
$z \mapsto \frac{1}{2} z^2$.
Now if $f: X\rightarrow Y$ and $g: Y\rightarrow Z$ are functions, then the chain rule says
$(g\circ f)'(x) = (g'\circ f)(x) \cdot f'(x)$,
which in this case means
$f'(\theta) = (y-N(x,\theta)) \cdot \left(-\frac{\partial N(x,\theta)}{\partial \theta}\right)$.
Edit: OP said that $x$ and $y$ also should be considered as functions of $\theta$. In this case, we get
$f'(\theta) = (y(\theta) - N(x(\theta),\theta)) \cdot (y'(\theta) - N'(x(\theta),\theta)) = (y(\theta) - N(x(\theta),\theta)) \cdot (y'(\theta) - \left(\frac{\partial N(x(\theta),\theta)}{\partial x} \cdot \frac{dx}{d\theta} + \frac{\partial N(x(\theta),\theta)}{\partial \theta}\right)$.
Here I used the notation with the prime to denote total derivatives.