$$ f(x,y) = \begin{cases} \dfrac{x^2 y^4}{x^4 + 6y^8}, & \text{if }(x,y)\neq(0,0) \\ 0, & \text{if }(x,y)=(0,0) \end{cases} $$
For the definition of differentiability, I have:
$$\lim_{h \rightarrow 0} \dfrac{||f(x+h, y+h) - f(x,y) - J(h)||}{||h||} = 0$$
So plugging in the function at $(0,0)$:
$$\lim_{h \rightarrow 0} \dfrac{\left\|\frac{(0+h)^2 (0+h)^4}{(0+h)^4 + 6(0+h)^8} - f(0,0) - J(h)\right\|}{||h||} = 0$$
Because $f(0,0)$ is defined to be 0 at that point:
$$\lim_{h \rightarrow 0} \dfrac{\left\|\frac{h^6}{h^4 + 6h^8} - J(h)\right\|}{||h||} = 0$$
I know $J(h)$ will be a $1 \times 2$ matrix, so I'm not sure how to deal with this situation here on out.
Hint: (i) Approach $(0,0)$ along the curve $x=y^2$; (ii) Approach $(0,0)$ along the curve $x=y$.