Knowing that $\sum_{n\leq x}\mu^2(n)=\frac{6}{\pi^2}x+O(\sqrt{x})$, prove that:
$$\sum_{n\leq x,\,\,n\equiv 1(\text{mod }p)}\mu^2(n)=\frac{6}{\pi^2(p-1)}x+O(\sqrt{x})$$
Using Dirichlet charaters, we have: \begin{align*} \sum_{n\leq x,\,\,n\equiv 1(\text{mod }p)}\mu^2(n)&=\frac{1}{\varphi(p)}\sum_{n\leq x}\sum_{\chi(\text{mod }p)}\mu^2(n)\chi(n)\\ &=\frac{1}{\varphi(p)}\sum_{\chi(\text{mod }p)}\sum_{n\leq x}\mu^2(n)\chi(n)\\ &=\frac{1}{\varphi(p)}\sum_{n\leq x}\mu^2(n)\chi_0(n)+\underbrace{\frac{1}{\varphi(p)}\sum_{\chi\neq \chi_0}\sum_{n\leq x}\mu^2(n)\chi(n)}_{=:\Delta(x)} \end{align*} Where $\chi_0$ is the principal character. I was able to prove that $\Delta(x)=O(\sqrt{x})$ basically by using the fact that $\mu^2(n)=\sum_{d|n}\mu(d)$, and that $\left|\sum_{n\leq x}\chi(n)\right|\leq \varphi(p)$. My problem is the main term, which doesn't match my calculation.
By definition, $\chi_0(n)=1$ if $p\not| n$ and $\chi_0(n)=0$ if $p|n$. Therefore: \begin{align*} \frac{1}{\varphi(p)}\sum_{n\leq x}\mu^2(n)\chi_0(n)&=\frac{1}{\varphi(p)}\sum_{n\leq x}\mu^2(n)-\frac{1}{\varphi(p)}\sum_{n\leq x,\,\,p|n}\mu^2(n)\\ &=\frac{1}{\varphi(p)}\sum_{n\leq x}\mu^2(n)-\frac{1}{\varphi(p)}\sum_{n\leq \frac{x}{p}}\mu^2(n)\\ &=\frac{1}{\varphi(p)}\left(\frac{6}{\pi^2}x-\frac{6}{\pi^2}\frac{x}{p}\right)+O(\sqrt{x})\\ &=\frac{6x}{\pi^2\varphi(p)}\left(1-\frac{1}{p}\right)+O(\sqrt{x})\\ &=\frac{6}{\pi^2p}x+O(\sqrt{x}) \end{align*} What am I missing?
In your treatment of the main term you have missed that $$\sum_{\substack{n \leqslant x \\ p \mid n}} \mu^2(n)$$ is not quite the same as $$\sum_{n \leqslant x/p} \mu^2(n) = Q\biggl(\frac{x}{p}\biggr)\,.$$ For large enough $x$, there are squarefree $n \leqslant x/p$ that are themselves divisible by $p$ and therefore do not give rise to a squarefree $m \leqslant x$ that is divisible by $p$. We have $$\sum_{\substack{n \leqslant x \\ p \mid n}} \mu^2(n) = \sum_{\substack{n \leqslant x/p \\ p \nmid n}} \mu^2(n)\,.$$ And we're back at the beginning, just with an upper limit of $x/p$ instead of $x$. Iterating the argument yields $$\sum_{\substack{n \leqslant x \\ p \mid n}} \mu^2(n) = \sum_{k = 0}^{\infty} (-1)^k Q\biggl(\frac{x}{p^k}\biggr) = \frac{6}{\pi^2}x\sum_{k = 0}^{\infty} \frac{(-1)^k}{p^k} + O(\sqrt{x}) = \frac{6}{\pi^2\bigl(1 + \frac{1}{p}\bigr)} x + O(\sqrt{x})$$ and hence $$\sum_{n \equiv 1 \pmod{p}} \mu^2(n) = \frac{6p}{\pi^2(p^2-1)}x + O(\sqrt{x})\,.$$ Thus the formula you were asked to prove is indeed wrong, but by a smaller factor (namely $\frac{p+1}{p}$) than you thought (namely $\frac{p}{p-1}$).