I try to show the following: The only point of $\operatorname{Spec}k[x,y]/(y^2,xy)$ with a nonreduced stalk is the origin.
The points in $\operatorname{Spec}k[x,y]/(y^2,xy)$ are $(y),(x-a,y)$ for $a\in k$. The localization $\operatorname{Spec}k[x,y]/(y^2,xy)_{(y)}=0$ is reduced. But I don't know how to determine the localization at $(x-a,y)$.
How can I determine what the localization $\operatorname{Spec}k[x,y]/(y^2,xy)_{(x-a,y)}$ is isomorphic to? I'm not sure in general how to determine a localization.
Here is an example using geometric interpretation: $\operatorname{Spec}k[x,y]/(xy)$ is the union of two axes in the plane. Localizing at $x$ means throwing out the locus where $x$ vanishes, so one expects $(\operatorname{Spec}k[x,y]/(xy))_{(x)}\cong\operatorname{Spec}k[x]_x$. Using this example, $\operatorname{Spec}k[x,y]/(y^2,xy)$ is the $x$-axis and localizing at $y$ is to throw away $x$ axis, so it's $0$. But I'm not sure how to deal with other cases. If I write $k[x,y]/(y^2,xy))_{(x-a,y)}=\{ \frac{f(x)+ay}{g(x)+by}: g(a)\neq 0\} $, then how can I proceed to determin this localization?
First, there's two different types of localizations: we can localize at an element, which means throwing out the vanishing locus of that element, or we can also localize at a prime ideal, which means looking "in a small neighborhood" of that prime ideal. They're very different, and your example in your final paragraph is of the first kind (and has some issues - you want to write $\operatorname{Spec} (k[x,y]/(xy))_x \cong \operatorname{Spec} k[x]_x$ but you appear to have made a typo or two) while you're trying to compute with the second kind in your problem.
Let's recall the definitions of localization. For a multiplicatively closed set $S$ of a ring $A$, we define the localization $S^{-1}A$ to be the ring of fractions $\frac{a}{s}$ with $a\in A$ and $s\in S$ with the usual addition and subtraction, where two fractions $\frac{a}{s}$ and $\frac{a'}{s'}$ are equal iff there is some $u\in S$ with $u(as'-a's)=0$. Localization at an element corresponds to setting $S=\{1,f,f^2,\cdots\}$, where localization at a prime ideal $\mathfrak{p}$ corresponds to setting $S=A\setminus\mathfrak{p}$.
In order to solve the problem, recall that localization is exact, so it commutes with quotients: $$(k[x,y]/(y^2,xy))_{(x-a,y)} \cong k[x,y]_{(x-a,y)}/(y^2,xy)_{(x-a,y)}.$$ Next, note that $a\neq 0$ is equivalent to $x\notin (x-a,y)$, so $x$ is invertible in this localization iff $a\neq 0$. When $x$ is invertible, then $(y^2,xy)=(y)$, and the quotient is just $k[x,y]_{(x-a,y)}/(y)_{(x-a,y)}\cong (k[x,y]/(y))_{(x-a,y)}\cong k[x]_{(x-a)}$ which is clearly reduced. When $a=0$, $x$ isn't invertible, and $y\neq 0$ but $y^2=0$ which means the ring $(k[x,y]/(y^2,xy))_{(x,y)}$ has nilpotents and isn't reduced.