I have a matrix which I can write
$$A_{ij} = \delta_{ij} + \xi^2 q_i q_j$$
which I wish to invert. I know the result is
$$A_{ij}^{-1} = \delta_{ij} - \frac{q_i q_j}{q^2 + \xi^{-2}}$$
I can easily verify that this is correct, but I don't know how to get this result from the original matrix.
Here Dirac's Braket notation helps. In that form your operator $A$ can be written as $$A = 1 + \xi^2 \lvert q\rangle \langle q \rvert.$$
Now $\langle q | q \rangle = \lVert q \rVert^2$ so for $k \geq 1$ you get $$\left(\lvert q\rangle \langle q \rvert\right)^k = \lVert q \rVert^{2(k-1)} \lvert q\rangle \langle q \rvert.$$
For small $\xi$ the inverse can now be written down as a geometric series:
$$\begin{eqnarray} \left(1 + \xi^2 \lvert q\rangle \langle q \rvert\right)^{-1} &=& \sum_{k=0}^{\infty}\left(-\xi^2 \lvert q\rangle \langle q \rvert\right)^k \\ &=& 1 +\lVert q \rVert^{-2}\sum_{k=1}^{\infty}(-\xi^2 \lVert q \rVert^2)^k \lvert q\rangle \langle q \rvert \\ &=& 1 - \xi^2 \sum_{k=0}^{\infty}(-\xi^2 \lVert q \rVert^2)^k \lvert q\rangle \langle q \rvert \\ &=& 1 - \xi^2\frac1{1+\xi^2 \lVert q \rVert^2} \lvert q\rangle \langle q \rvert \\ &=& 1 - \frac1{\xi^{-2}+\lVert q \rVert^2} \lvert q\rangle \langle q \rvert \end{eqnarray} $$