I would like to count the elements of $\mathbb{Z}[\sqrt{2}]$ modulo its units, so I'll remove the set $\{ 0\}$ but also the group of units $\{ (1 + \sqrt{2})^n : n \in \mathbb{Z}\}$. So this would be the quotient set $\mathbb{Z}[\sqrt{2}]^\times / (1 + \sqrt{2})^\mathbb{Z} $.
So I am going to place my copy of $\mathbb{Z}[\sqrt{2}]$ in the plane using:
$$ \mathbb{Z}[\sqrt{2}] = \{ (a,b\sqrt{2}): a,b \in \mathbb{Z} \}\hookrightarrow \mathbb{R}^2 $$
One possible lattice point counting problem could be a circle or an ellipse, we could count elements of the set:
$$ \{ x^2 + y^2 < R^2\} \cap \{ (a,b\sqrt{2}): a,b \in \mathbb{Z} \} $$
This does not seem natural since the norm for $\mathbb{Q}(\sqrt{2})$ has to do with hyperbolas:
$$ N(a + b \sqrt{2}) = a^2 - 2b^2 = n $$ where $n \in \mathbb{Z}$ ranges over the integers. This is a family of hyperbolas, preserved under the action of $ \times \,(\,1 + \sqrt{2})$.
In that case, what are the quotient sets of $\mathbb{C}^\times $ modulo the action of $ \times \,(\,1 + \sqrt{2})$ ?
I have drawn the folation and transverse foliations of the hyperbola, which cover the pictured Euclidean plane:
\begin{eqnarray*} x^2 - 2y^2&=& a \\ \sqrt{2} \, xy&=& b \end{eqnarray*}
Here the element $1 + \sqrt{2}$ acts as:
$$ (1 + \sqrt{2}) (a + b \sqrt{2}) = (a + 2b) + \sqrt{2}(a+b) $$
and this can be modeled as a $2 \times 2$ matrix
$$ 1 + \sqrt{2} \leftrightarrow \left[ \begin{array}{cc} 1 & 2 \\ 1 & 1 \end{array} \right] $$
Back to the original problem I'd liked to list elements of $\mathbb{Z}[\sqrt{2}]$ that are "smaller than" say 100 up to the action of the unit $1 \pm \sqrt{2}$. This has motivated me to define some peculiar sets of the plane, but I still cannot get an answer.
Hopefully I have explained what I am looking for.
This is more of a comment, but it is too long for the comment box.
The embedding $\mathbb{Z}[\sqrt{2}] \hookrightarrow \mathbb{R}^2$ that you picked is not the most convenient one for studying this type of problem, as your comment about norms suggests.
There is a much better embedding, namely $$a + b \sqrt{2} \mapsto (a + b \sqrt{2},a - b \sqrt{2}) $$ This embedding has many advantages in connection with your problem: like the embedding you chose, its image is discrete: the norm is given by the simple formula $|(x,y)| = xy$; both the addition and the multiplication are operators given by coordinate-wise operation on vectors; multiplication by any particular unit is given by a simple diagonal matrix in $(x,y)$ coordinates.
This is discussed in many elementary number theory books -- there is a similar discrete embedding for the ring of integers of any number field. I suspect your counting problem will be simplified by using this point of view.