I've seen this note and got more improved understanding of how relative extrema could be found in calculus. Professor Strang in his linear algebra course explains how function could be minimized by orthogonal projections and brings the alternative way of finding the local minimum, in which he mentions differentiation.
Say we have linear function ($mx + c$) ($m$ being slope, the first variable and $c$ being bias, the second variable, both of these are 0 dimensional, therefore they are scalars) which contains two variables, it's squared error function is $||e||^2$. In order to find local minimum of this error function derivatives must be used, since every relative extrema is a critical point and critical point $x$ only exists when $f'(x) = 0$ or $f'(x)\uparrow$. Considering that we are using linear function with two variables, partial derivatives must be used to solve minimization problem for both variables.
In multivariable case, critical point of function $f(x, y)$ exists when $\triangledown f(x, y)=\vec 0$ or $\triangledown f(x, y)\uparrow$.
After using this method on the error function of my linear function, I've got the local minimum multiplied by two (for the reason that I'm currently not aware of), but why did it work?
I understand local minimum must be the critical point which is stationary point and therefore it's derivative is zero, but there may be other critical points in the function too.
Why didn't I get relative maximum or a saddle point? they are critical points too. May it be because of my error function constructed for minimization purposes?
You are minimizing a linear least squares objective, which is a convex function. Therefore, every critical point is a global minimum. That's all there is to it.