Let $f: \mathbb{R}^2\to \mathbb{R}$ be defined by setting
$$f(x,y)=\begin{cases}\frac{x^2y^2}{x^2y^2+(y-x)^2} & (x,y)\neq (0,0)\\0 & (x,y)=(0,0)\end{cases}$$ (a) For which vectors $u\ne 0$ does $f'(0; u)$ exist? Evaluate it when it exists.
(b) Do $D_1f$ and $D_2f$ exist at $0$?
(c) Is $f$ differentiable at $0$?
(d) Is $f$ continuous at $0$?
I have thought a lot about this problem:
For (a), let $u\neq 0$ and $u=(h,k)$ then $\lim_{t\to 0}\frac{f(0+tu)-f(0)}{t}=\lim_{t\to 0}\frac{f(0+t(h,k))-f(0)}{t}=\lim_{t\to 0}\frac{th^2k^2}{t^2h^2k^2+t(k-h)^2}$ but I do not know what conditions $h$ and $k$ have to satisfy so that this limit exists, could someone help me please?
For (b), since $f(t,0)=0$ and $f(0,t)=0$ it is concluded that $D_1f(0,0)=\lim_{t\to 0}\frac{f(t,0)-f(0)}{t}=0$ and $D_2f(0,0)=\lim_{t\to 0}\frac{f(0,t)-f(0)}{t}=0$ thus these limits exist.
For (c), we see that taking the director vector $u=(1,1)$ and doing $\lim_{t\to 0}\frac{f(0+tu)-f(0)}{t}=\lim_{t\to 0}\frac{1}{t}$ we see that the limit does not exist and thus the function is not differentiable in $0$
For (d), I am having trouble solving this, I am trying to calculate $\lim_{(x,y)\to 0, y=mx}f(x,y)$and I get to that $\lim_{(x,y)\to 0, y=mx}f(x,y)=\lim_{(x,y)\to 0, y=mx}\frac{x^2(mx)^2}{x^2(mx)^2+(mx-x)^2}$,I do not know if $f$ is continuous in $0$, could someone help me please?
To show that the limit does not exist approach $(0,0)$ along $y=0$ and along $y=x$ you get two differnt limits $0$ and $1$. This implies it is not continuous and hence not differentiable at $(0,0)$. For the directional derivative you didn't put the vector $u=(u_1,u_2)$ into the equation. You have to compute $$\displaystyle\lim_{t\to 0}\frac{f((0,0)+t(u_1,u_2))-f(0,0)}{t}=\lim_{t\to 0}\frac{f(tu_1,tu_2)-f(0,0)}{t}$$$$=\lim_{t\to 0}\frac{u_1^2u_2^2t^4}{t^5u_1^2u_2^2+t^3(u_1-u_2)^2}=\lim_{t\to 0}\frac{u_1^2u_2^2t}{t^2u_1^2u_2^2+(u_1-u_2)^2}=0,$$ for any unit vector $(u_1, u_2)$ with $u_1\neq u_2$ .