Let $A$ a positive-definite $n \times n$ matrix with eigenvalues in the interval $[a, b]$ and let $x=(x_1,\ldots,x_n)$ be a random vector with iid components distributed uniformly in $\{\pm 1\}$.
Question. What is a nontrivial lower-bound for $\mathbb E\|A^{-1}x\|_2$ ?
Observations
$\|A^{-1}x\| \ge \lambda_\min(A^{-1})\|x\| \ge \|x\|/b=\sqrt{n}/b$ and so $\mathbb E[\|A^{-1}x\| \ge \sqrt{n}/b$, a bound which can only be naive, since it doesn't exploit the fact that $x$ is random.
$n = \|x\|^2 = x^\top x = x^\top AA^{-1} x \le \|Ax\|\|A^{-1} x\| \implies \|A^{-1}x\| \ge \dfrac{n}{\|Ax\|}$. On the other hand, for any $R>0$, Markov's inequality gives $$ R\cdot \mathbb E[\|Ax\|^{-1}] = \dfrac{\mathbb E[\|Ax\|^{-1}]}{R^{-1}} \ge \mathbb P(\|Ax\|^{-1} \ge R^{-1}) = \mathbb P(\|Ax\| \le R). $$ Thus, $\mathbb E[\|A^{-1}x\|] \ge \dfrac{n}{R} \cdot \mathbb P(\|Ax\| \le R)$.
As (almost) noted in the comments, you have an exact value for this quantity:
\begin{align*} \mathbb{E}\big(||A^{-1}x||_2^2\big) & = \mathbb{E}\big(x^T A^{-2} x) = \mathbb{E}\Big(\sum \limits_{1 \le i,j \le n} (A^{-2})_{i,j}x_ix_j\Big) = \sum \limits_{1 \le i,j \le n} (A^{-2})\cdot \mathbb{E}(x_ix_j) = \sum \limits_{i=1}^n (A^{-2})_{i,i} \end{align*}
since $\mathbb{E}(x_ix_j)=0$ if $i \neq j$ and $\mathbb{E}(x_i^2) = \mathbb{E}(1) = 1$.
Hence $\mathbb{E}\big(||Ax||_2^2\big) = Tr\big(A^{-2}\big)$. If you wish, you can write that last expression as the sum of the inverses of the squared eigenvalues of $A$.