I'm studying multivariable calculus and we defined a derivative of a function $f(x)$ from $\mathbb{R}^m$ to $\mathbb{R}^n$ to be a linear operator $A'$ such that
$$\lim_{h\to 0} \frac{|f(x+h)-f(x)-A'h|}{|h|} = 0$$
The simplest example may be the the linear transformations $A$, because by their linearity, $A(x+h) = Ax + Ah \implies A(x+h) -Ax = Ah$, therefore, in the limit definition, if we choose $A' = A$, we have
$$\lim_{h\to 0} \frac{|A(x+h) -Ax-A'h|}{|h|} =\lim_{h\to 0} \frac{0}{|h|} = 0$$
therefore, $A' = A$ because the derivative is unique and we just found one that worked.
Now the hard part comes when we need to find a derivative of a function that is not linear. For example:
$$f:\mathbb{R}^3\to \mathbb{R}^2\\f((x_1,x_2,x_3)) = (x_1^2, x_3^2)$$
I need to find $f':\mathbb{R}^3\to \mathbb{R}^2$ that satisfies
$$\lim_{h\to 0} \frac{|f(x+h)-f(x)-A'h|}{|h|} = \lim_{h\to 0} \frac{|((x_1+h_1)^2, (x_3+h_3)^2) - (x_1^2, x_3^2) - (x_1', x_2')|}{|(h_1,h_2,h_3)|} = 0$$
Where $A'h = (x_1', x_2')$
Let's expand the limit:
$$ \lim_{h\to 0} \frac{|(x_1^2 + 2x_1h_1 + h_1^2, x_3^2 + 2x_3h_3 + h_3^2) - (x_1^2, x_3^2) - (x_1', x_2')|}{\sqrt{h_1^2 + h_2^2 + h_3^2}} = \\ \lim_{h\to 0} \frac{|(2x_1h_1 + h_1^2, 2x_3h_3 + h_3^2) - (x_1', x_2')|}{\sqrt{h_1^2 + h_2^2 + h_3^2}}$$
The hard part is now to take $x_1', x_2'$, that is, the rule of $A'$ that will make it work. Below is something I tried but then I realized I can't even do that (write $x_1', x_2'$ in terms of $x_1, x_2, x_3$)
If I pick: $x_1'= 2x_1h_1$ and $x_2'= 2x_3h_3$ we end up with
$$\lim_{h\to 0} \frac{|(h_1^2, h_3^2)|}{\sqrt{h_1^2 + h_2^2 + h_3^2}} = \lim_{h\to 0}\frac{\sqrt{h_1^4 + h_3^4}}{\sqrt{h_1^2 + h_2^2 + h_3^2}}$$
How should I find $A'$?
In this case $A'$ is the jacobian matrix and $h=(h_1,h_2,h_3)$, notably we have
$$A'=\begin{bmatrix}2x_1&0&0\\0&2x_2&0\end{bmatrix}$$
Indeed we have that
$$f(x_1+h_1,x_2+h_2,x_3+h_3)=((x_1+h_1)^2,(x_2+h_2)^2)=$$
$$=(x_1^2+2x_1h_1+h_1^2,x_2^2+2x_2h_2+h_2^2)=(x_1^2,x_2^2)+(2x_1h_1,2x_2h_2)+ (h_1^2,h_2^2) =$$
$$=f(x_1,x_2,x_3)+A'(x_1,x_2,x_3)\cdot (h_1,h_2,h_3)+(h_1^2,h_2^2)$$
with
$$A'(x_1,x_2,x_3)\cdot (h_1,h_2,h_3)=\begin{bmatrix}2x_1&0&0\\0&2x_2&0\end{bmatrix}\begin{bmatrix}h_1\\h_2\\h_3\end{bmatrix}=(2x_1h_1,2x_2h_2)$$
and
$$\lim_{h\to 0}\frac{(h_1^2,h_2^2)}{\sqrt{h_1^2+h_2^2+h_3^2}}\to(0,0)$$