How do I calculate the partial derivative of this function?

Question

$$ J(\theta) = - \frac{1}{m} [ \sum_{i=1}^{m} y^i log( h_\theta(x^i) ) + (1-y^i) log( 1 - h_\theta(x^i))] $$

This is the equation of the cost function for logistic regression. To apply gradient descent to it, I need to calculate partial derivatives with respect to $\theta_j$.

How do I calculate $\frac{\partial}{\partial\theta_j} J(\theta)$ ?

Edit: I am thinking it'll be along the lines of product rule of derivatives but I am unable to work it out.

Answer 1

I assume that $h_{\theta}(x^i)$ is the only part of each term that depends on $\theta_j$. We then have \begin{align}\frac{\partial }{\partial \theta_j}J(\theta)&=-\frac{1}{m}\sum_{i=1}^m \frac{y^i} {h_{\theta}(x^i)}\frac{\partial}{\partial \theta_j}h_{\theta}(x^i)-\frac{1-y^i}{1-h_{\theta}(x^i)}\frac{\partial}{\partial \theta_j}h_{\theta}(x^i) \\ &=-\frac{1}{m}\sum_{i=1}^m \frac{\partial}{\partial \theta_j}h_{\theta}(x^i)\left(\frac{y^i} {h_{\theta}(x^i)}-\frac{1-y^i}{1-h_{\theta}(x^i)}\right) \end{align}

Here I have used the chain rule and that $\frac{\partial \ln(x)}{\partial x}=\frac{1}{x}$.

Without any further specification of exactly how $J(\theta)$ depends on $\theta_j$, it is impossible for me to go any further with the calculations.

Answer 2

I explained this Here.However, follow the solution.

$$ J(\theta) = - \frac{1}{m} \sum_{i=1}^m [y^{(i)}\ln (h_\theta (x^{(i)})) + (1 - y^{(i)})\log (1 - h_\theta(x^{(i)}))] $$

Below is the explanation that how to derivative the Cross-Entropy.

$$ \dfrac{dJ(\theta)}{dy} = \dfrac{1}{m}. \Big[\dfrac{y}{h_\theta (x^{(i)})}-\dfrac{(1-y)}{(1-h_\theta (x^{(i)}))}\Big] $$

$$ = \dfrac{1}{m}. \Big[\dfrac{y.(1-h_\theta (x^{(i)})) - (1-y).h_\theta (x^{(i)})}{h_\theta (x^{(i)}).(1-h_\theta (x^{(i)}))}\Big] $$

$$ = \dfrac{1}{m}. \Big[\dfrac{y-yh_\theta (x^{(i)}) - h_\theta (x^{(i)})+y h_\theta (x^{(i)})}{h_\theta (x^{(i)}).(1-h_\theta (x^{(i)}))}\Big] $$ $$ = \dfrac{1}{m}. \Big[\dfrac{y - h_\theta (x^{(i)})}{h_\theta (x^{(i)}).(1-h_\theta (x^{(i)}))} \Big] $$