Reading this :
From http://www.holehouse.org/mlclass/09_Neural_Networks_Learning.html
Cost function for a single training example is given as :
cost(i) = $ y^i \; log \; h_\theta(x^i) + (1 - y^i) log \; h_\theta (x^i)$
Are my steps to work out the cost for a single training example correct ? :
cost(1) = $ y^1 \; log \; h_\theta(x^1) + (1 - y^1) log \; h_\theta (x^1)$
$ x^1 = $
\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}
$ y^1 = $
\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}
$ h_\theta(x) = 1 / 1+e{^-\theta Tx} $
Working this out :
Randomly initialise $\theta$ values to \begin{bmatrix} .1 \\ .1 \\ .1 \end{bmatrix}
cost(1) = $ y^1 \; log \; (1 / 1 + e - ^{\begin{bmatrix} .1 \\ .1 \\ .1 \end{bmatrix}^T \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} } )+ (1 - y^1) log \; (1 / 1 + e - ^{\begin{bmatrix} .1 \\ .1 \\ .1 \end{bmatrix}^T \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} } )$
Substituting is this correct ? :
cost(1) = $ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \; log \; (1 / 1 + e - ^{\begin{bmatrix} .1 \\ .1 \\ .1 \end{bmatrix}^T \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} } )+ (1 - \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}) log \; (1 / 1 + e - ^{\begin{bmatrix} .1 \\ .1 \\ .1 \end{bmatrix}^T \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} } )$
cost(1) = $ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \; log \; (1 / 1 + e - ^.2 )+ (1 - \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}) log \; (1 / 1 + e - ^.2 )$
cost(1) = $ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \; log \; (1 / 1 + .82 )+ (1 - \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}) log \; (1 / 1 + .82 )$
cost(1) = $ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \; log \; .549+ (1 - \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}) log \; .549$
cost(1) = $ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \; -.26+ \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} -.26$
cost(1) = $ \begin{bmatrix} .74 \\ .74 \\ -.26 \end{bmatrix} \;+ \begin{bmatrix} -.26 \\ -.26 \\ .74 \end{bmatrix}$
cost(1) = $ \begin{bmatrix} .48 \\ .48 \\ .48 \end{bmatrix}$