$\newcommand{\f}[2]{\frac{#1}{#2}}$ $\newcommand{\nor}[2]{ \left| \! \left| #1 \right| \! \right|_{#2} }$ $\newcommand{\ab}[1]{\left|#1\right|}$ $\newcommand{\R}{\mathbb{R}}$ $\newcommand{\pa}[1]{\left( #1 \right)}$ $\def\1{1}$
Consider the norm \begin{align*} \nor{V}{L^{\f{3}{2}}(\R^3) + L^\infty(\R^3)} := {\underset{\substack{V_{3/2} \in L^{\f{3}{2} }(\R^3) \\ V_{\infty} \in L^\infty(\R^3) \\ V = V_{3/2} + V_\infty}}{\text{inf}}\quad} \pa{\nor{V_{3/2}}{L^{\f{3}{2}}} + \nor{V_\infty}{L^\infty(\R^3)}} \end{align*} which defines a Banach space. It enables to treat local singularities since $\ab{\cdot}^{-p} \in L^{\f{3}{2}} + L^\infty$ for any $p < 2$, by decomposing $\ab{\cdot}^{-p} = \ab{\cdot}^{-p} \1_{\ab{\cdot} \le c} + \ab{\cdot}^{-p}\1_{\ab{\cdot} \ge c}$ (the first function is in $L^{\f{3}{2}}$ and the second in $L^{\infty}$) but $\ab{\cdot}^{-p} \notin L^{\f{3}{2}}$. But $\ab{\cdot}^{-2} \notin L^{\f{3}{2}} + L^\infty$. My question is, do we have \begin{align*} \nor{ \ab{r-\cdot}^{-1} - \ab{\cdot}^{-1} }{L^{\f{3}{2}} + L^\infty} \le c \ab{r} \end{align*} for any $r \in \R^3$, where $c$ does not depend on $r$ ? Of course it's when $r$ is small that there is an issue.
At first sight, it is natural to do \begin{align}\label{eqq} \ab{\ab{r-x}^{-1} - \ab{x}^{-1}} &= \ab{\ab{r-x} - \ab{x}}\ab{r-x}^{-1}\ab{x}^{-1} \nonumber \\ & \le \ab{r} \ab{r-x}^{-1}\ab{x}^{-1} \end{align} but then $\nor{\ab{r-\cdot}^{-1}\ab{\cdot}^{-1}}{L^{\f{3}{2}} + L^\infty}$ is not bounded in $r$ when $r$ is small, since \begin{align*} \nor{\ab{r-\cdot}^{-1}\ab{\cdot}^{-1}}{L^{\f{3}{2}} + L^\infty} \rightarrow +\infty \end{align*} when $\ab{r} \rightarrow 0$. But for any $\varepsilon > 0$ small, \begin{align*} \ab{r}^\varepsilon \nor{\ab{r-\cdot}^{-1}\ab{\cdot}^{-1}}{L^{\f{3}{2}} + L^\infty} \rightarrow 0 \end{align*} So is there an approach, which is finer and which yields the result ? Or what is the lowest singularity, that we cannot remove ?
As a summary, there exists a constant $C>0$ such that for any $z\in\mathbb R^3$, $$ \left\|\frac{1}{|x-z|} - \frac{1}{|x|}\right\|_{L^{3/2}_x+L^\infty_x} \leq C\,|z|\left(1+(-\ln(|z|))_+^{2/3}\right) $$ with $a_+ = \max(a,0)$. If you do not want logarithms to appear, you can consider differences of order $2$, and then by the references indicated in my comment, there exists a constant $C>0$ such that for any $z\in\mathbb R^3$, $$ \left\|\frac{1}{|x-z|} + \frac{1}{|x+z|} - \frac{2}{|x|}\right\|_{L^{3/2}_x} \leq C\,|z| $$
Proof of the first formula. If you do not like complicated spaces, the best strategy in this kind of problems is to cut the integral. One can write $|x|^{-1} = K(x) + \varphi(x)$ with $\varphi(x) = \min(|x|^{-1},1)$ is a bounded Lipschitz function, and $K(x) = (|x|^{-1}-1)_+$. More precisely, $|\nabla\varphi|\leq 1$ so $\|\varphi(\cdot-z)-\varphi\|_{L^\infty}\leq |z|$. The difficulty is to handle $K$. Let me look at $K(x-z)-K(x)$ when $|z|\leq 1/2$, since as you say, the difficulty is when $|z|$ is small. Then by the triangle inequality $$ I_K := \|K(\cdot-z)-K\|_{L^{3/2}} \leq \left(\int_{|x|\leq 1 \,\&\, |x-z|\leq 1} \left|\frac{1}{|x-z|}-\frac{1}{|x|}\right|^{3/2}\mathrm d x\right)^{2/3} \\ + \left(\int_{|x|\geq 1 \,\&\, |x-z|\leq 1} K(x-z)^{3/2}\mathrm d x\right)^{2/3} + \left(\int_{|x-z|\geq 1 \,\&\, |x|\leq 1} K(x)^{3/2}\mathrm d x\right)^{2/3}. $$ One can do a change of variable $x\mapsto z-x$ in the second integral of the right hand side to get $$ I_K \leq \left(\int_{|x|\leq 1} \left|\frac{1}{|x-z|}-\frac{1}{|x|}\right|^{3/2}\mathrm d x\right)^{2/3} + 2\left(\int_{|x-z|\geq 1 \,\&\, |x|\leq 1} \left(\frac{1}{|x|}-1\right)^{3/2}\mathrm d x\right)^{2/3}. $$ Now let me treat each of these two integrals.
The first integral
Putting these estimates together, we get that the first integral is bounded by $$ (J_r^c)^{2/3} + J_r^{2/3} \leq (4\pi)^{2/3} \left(5\,(2/3)^{2/3} + 2 \,\left|\ln(2\,|z|)\right|^{2/3}\right) |z|. $$
The second integral
Conclusion
For any $|z|\leq 1/2$ $$ \||\cdot-z|^{-1}-|\cdot|^{-1}\|_{L^\infty+L^{3/2}} \leq \|\varphi(\cdot-z)-\varphi\|_{L^\infty} + \|K(\cdot-z)-K\|_{L^{3/2}} \\ \leq \left(1 + \left(\tfrac{7\,\pi}{6}\right)^{2/3} + (4\pi)^{2/3} \left(5\,(2/3)^{2/3} + 2 \,\left|\ln(2\,|z|)\right|^{2/3}\right)\right) |z|. $$ The constants are not at all optimal of course. To simplify, this implies that for any $|z|\leq 1/2$ $$ \||\cdot-z|^{-1}-|\cdot|^{-1}\|_{L^\infty+L^{3/2}} \leq 25 \left(1 + \,\left|\ln(|z|)\right|^{2/3}\right) |z|. $$