Closed Subschemes of an affine scheme are determined by ideals $\mathfrak{a}\subseteq \Gamma(X,\mathcal{O}_X)$.
Consider the affine plane $\mathbb{A}^2=\text{Spec} k[x,y]$ then the ideal $(x,y)$ defines the origin as a closed point.
$\bf{Question\ 1:}$ What subschemes are defined by $(x^2,y)$ and $(x,y^2)$ ?
Some thoughts: The subscheme $(x^2,y)$ lies on the $x$-axis as $(y)\subseteq (x^2,y)$, thus in analogy with the idea that $(x^2)\subseteq k[x]$ defines an infinitesimal neighbourhood of the point $0$, we have that $(x^2,y)$ is the same infinitesimal neighbourhood of a point on a line but now embedded in the plane. Similarly, $(x,y^2)$ lies on the $y$ axis.
$\bf{Question\ 2:}$ Gives a closed subscheme $\mathfrak{a}$ can one determine a reduced irreducible subscheme on which it lies ?
Some more thoughts, the subscheme $(x^2,xy,y^2)$ is the infinitesimal neighbourhood of the origin in the plane. But what about $(x^2,y^2)$ ? It does not seem to lie on some closed subscheme.
$\bf{Question\ 3:}$ What geometric interpretation can we give to the closed subscheme defined by the ideal $(x^2,y^2)$ ?
Thanks in advance for sharing your thoughts.
$\bullet$ Your remarks on Question 1 are quite correct.
$\bullet \bullet$As to Question 2:
Neither the subscheme $V_1=V(x^2,y^2)\subset \mathbb A^2$ nor $V_2=V(x^2,xy,y^2)\subset \mathbb A^2$ can lie on a smooth curve $C$, whether that curve is embedded in $\mathbb A^2$ or not.
The reason is that the Zariski tangent space of the scheme $V_i$ (at its unique point!) is $2$- dimensional and thus is not a subspace of the $1$-dimensional Zariski tangent space of a smooth curve at any of its points $c\in C$.
However both of these schemes are subschemes of plenty of singular but reduced and integral curves: for example $V_1,V_2\subset V(y^2-x^3)\subset \mathbb A^2$.
$\bullet \bullet \bullet$ Question 3 has many answers.
The simplest might be that if you restrict some function $$F(X,Y)=a+bX+cY+dXY+eX^2+gY^2+\cdots\in \mathcal O( \mathbb A^2)=k[X,Y]$$ to $V_1$, you will obtain a function $$f(x,y)=a+bx+cy+dxy\in \mathcal O( V_1)=k[X,Y]/(X^2,Y^2)=k[x,y]$$ which remembers not only the value $F(0,0)=a$ of $F$ at the origin but also the derivative of $F$ in any direction: $$ \partial F_{(0,0)}(u,v)=bu+cv$$ Algebraic geometry is much closer to Advanced Calculus than is often realised!