I'm having trouble seeing the relationship in the following equation. Let's assume $J(0,1)$ and $m=4$.
First I figure out my hypothesis function $$\begin{array}{rcl}h_\Theta(x)&=&\Theta_0+\Theta_1x\\&=&0+1x\\&=&x\end{array}$$
So my cost function should look like
$$J(\Theta_0,\Theta_1)=\frac{1}{2\times 4}\sum_{i=1}^m\left[h_\Theta(x^{(i)})-y^{(i)}\right]^2$$
This is where I become confused. Since our $h_\Theta(x)$ is $x$, isn't our $h_\Theta(x^{(i)})$ just $x^{(i)}$? So why isn't it: $$\begin{array}{rcl}J(\Theta_0,\Theta_1)&=&\frac{1}{8}[(1-1)^2+(2-2)^2+(3-3)^2+(4-4)^2]\\&=&\frac{1}{8}[0^2+0^2+0^2+0^2]\\&=&0\end{array}$$
I know the right way to do it is: $$\begin{array}{rcl}J(\Theta_0,\Theta_1)&=&\frac{1}{8}(1^2+1^2+1^2+1^2)\\&=&\frac{1}{8}\times 4\;=\;\frac{4}{8}\\&=&0.5\end{array}$$
Could someone explain to me how we get $1^2+1^2+1^2+1^2$.
Thanks!
I think You are confusing the notation. The super-index $n^{(i)}$ denotes the $i$-th element of your training set. For instance, you have the experimental data of $m$ pairs $$\begin{array}{l|lllll}x^{(i)}&x^{(1)}&x^{(2)}&\;\dotso\;&x^{(m)}\\\hline y^{(i)}&y^{(1)}&y^{(2)}&\;\dotso\;&y^{(m)}\end{array}$$ The hypothesis function does not compute the ouput $y^{(i)}$, but it computes the "approximated" value to $y^{(i)}$.
You have $(1−1)^2+(2−2)^2+(3−3)^2+(4−4)^2$ if $x^{(i)}=i$ and $y^{(i)}=i$. But this is a trivial training set. By other hand, you obtain $1^2+1^2+1^2+1^2$ if you data looks like $$\begin{array}{l|lllll}x^{(i)}&a&a&\;\dotso\;&a\\\hline y^{(i)}&a+1&a+1&\;\dotso\;&a+1\end{array}$$ or $$\begin{array}{l|lllll}x^{(i)}&a+1&a+1&\;\dotso\;&a+1\\\hline y^{(i)}&a&a&\;\dotso\;&a\end{array}$$ for some value $a$.
Note that this is true if your hypothesis function is still $h_\Theta(x) = x$.