I'm trying to show that there is no solution of this equation:
$x^2+\frac{7}{2}y^2=1-\frac{1}{\sqrt{2}}$
where $x,y\in \mathbb{Z}[\frac{1}{\sqrt{2}}]=\{n+\sqrt{2}m:n,m\in\mathbb{Z}[\frac{1}{2}]\}$.
I guess if there is a solution, then there are only finitely many solutions. Anyway, it may seem very elementary to number theorists... could you recommend me a good reference for this kind of problems?
The problem with talking about "this kind of problem," is that there are many different ways to think about it.
For example, one might think of the LHS as a norm equation
$$\displaystyle{N_{L/K}(u) = 1 - \frac{1}{\sqrt{2}}}$$
where $L = \mathbf{Q}(\sqrt{-7},\sqrt{2})$ and $K = \mathbf{Q}(\sqrt{2})$, and then ask if there are solutions. The solutions will be S-units in $L$, where $S$ denotes the set of primes above $2$. Here's a general way of solving this type of problem:
Compute the unit group of $\mathcal{O}_K[1/2]$. By Dirichlet's S-unit theorem, it has rank $2$. Here Dirichlet's S-unit theorem says that the rank of this group is $r_1 + r_2 - 1$ plus the number of primes in $S$. This is an easy consequence of the usual Dirichlet's formula plus the finiteness of the class group. Since $\mathcal{O}_K$ is a PID, the $2$-units are generated by $\sqrt{2}$, by the fundamental unit $\sqrt{2} - 1$ of $K$, and by the root of unity $-1$. This group is isomorphic to $\mathbf{Z}^2 \oplus \mathbf{Z}/2\mathbf{Z}$, and the RHS above is the element $(-1,1,0)$.
Compute the unit group of $\mathcal{O}_L[1/2]$. In this case the rank is $3$, since $r_1 + r_2 - 1 = 0+2-1 = 1$ and there are two primes above $2$. The unit group is still generated by $\sqrt{2} - 1$ and $-1$. There is a factorization
$$(\sqrt{2}) \mathcal{O}_L = \mathfrak{p} \mathfrak{q},$$
but $\mathfrak{p}$ and $\mathfrak{q}$ have order $2$ in the class group of $L$ (which has class number $2$). So we need to take the subgroup of $\mathfrak{p}^{\mathbf{Z}} \mathfrak{q}^{\mathbf{Z}}$ consisting of principal ideals. This subgroup has index two and is generated by $\mathfrak{p} \mathfrak{q} = (\sqrt{2})$ and $\mathfrak{p}^2 = (\eta)$, where
$$\eta = \frac{1 + \sqrt{-7}}{2}.$$
So write the $2$-unit group as $\mathbf{Z}^3 \oplus \mathbf{Z}/2 \mathbf{Z}$ with generators $\sqrt{2}$, $\sqrt{2} - 1$, $\eta$, and $-1$.
Now, after all of this, the norm map, in our basis, takes all the elements in $K$ to their squares, and takes the element $\eta$ to $-2$. Or, on our basis:
$$(1,0,0,0) \mapsto (2,0,0) \in \mathbf{Z}^2 \oplus \mathbf{Z}/2\mathbf{Z},$$ $$(0,1,0,0) \mapsto (0,2,0),$$ $$(0,0,0,1) \mapsto (0,0,0),$$ $$(0,0,1,0) \mapsto (2,0,-1).$$
So, by inspection, the element $(-1,1,0)$ is not in the image, and hence there are no solutions.
Note that for equations of this type, there may be no solutions, or finitely many solutions, or infinitely many solutions. In fact, if the rank of the source group is larger than the rank of the image (as it is in your case), then there will be either zero or infinitely many solutions (so your intuition that there would be finitely many was not quite correct).
For example, if the equation was
$$x^2 + \frac{7}{2} y^2 = 1,$$
then there would be infinitely many solutions. From our computation above, we see that solutions $(x,y)$ occur precisely when $x + \sqrt{-7}(y/\sqrt{2})$ is plus or minus a power of
$$\frac{\eta}{2} = \frac{1 + \sqrt{-7}}{2 \sqrt{2}}.$$
For example, taking the $10$th power, we get
$$\left(\frac{57}{64}\right)^2 + \frac{7}{2} \left(\frac{11}{32 \sqrt{2}}\right)^2 = 1,$$
or taking the $11$th power,
$$\left(\frac{67}{64 \sqrt{2}}\right)^2 + \frac{7}{2} \left(\frac{23}{64 }\right)^2 = 1.$$
Of course, in any particular situation, there are often shortcuts, and it's always a standby in number theory to ask "does this equation have solutions modulo $p$ for some small prime $p$?" This is a great way to show that there are no solutions, although it doesn't even work even when there are no solutions. Happily, such an argument works in this case, leading to a much shorter proof:
In the ring $R = \mathbf{Z}[1/\sqrt{2}]$, the ideal $\mathfrak{p} = (3 - \sqrt{2})$ is prime, and $R/\mathfrak{p} = \mathbf{Z}/7 \mathbf{Z}$. But then reducing modulo $7$, we get the equation
$$x^2 \equiv 1 - 3^{-1} \mod 7 = 1 - 5 \mod 7 = 3 \mod 7.$$
But $3$ is not a prime modulo $7$, so there are no solutions.