When proving the theorem of prime numbers, the professor introduced a function $\phi(s)=\sum_{p}\frac{\log p}{p^s}$, where $p$ is all prime numbers. After a calculation, we get this formula: $$-\frac{\zeta'(s)}{\zeta (s)}=\phi (s)+\sum_p \frac{\log p}{p^s (p^s-1)}.$$
Then the professor said that $\frac{\zeta'(s)}{\zeta(s)}$ has pole at $0$, and $Res(\frac{\zeta'(s)}{\zeta(s)},1)=-1$. He did not prove this and I don't know how to get it. Can anyone help me? Thanks very much!
On
When $f$ is a non constant meromorphic function defined in an open domain $U \subset \mathbb{C}$, you can write near each point $a \in U$ such that $f(a) = 0$ or $\infty$, $f(x) = \alpha(x - a)^n + \circ((x - a)^n)$ where $\alpha \in \mathbb{C}^*$ and $n \in \mathbb{Z}$. $n$ is the order of vanishing of $f$ at $a$.
In this case, $n \neq 0$ because $f$ has a zero or a pole at $a$ and, $$ \frac{f'(x)}{f(x)} = \frac{n\alpha(x - a)^{n - 1} + \circ((x - a)^{n - 1})}{\alpha(x - a)^n + \circ((x - a)^n)} = \frac{n}{(x - a)} + \circ((x - a)^{-1}) $$ We deduce that $f'/f$ has a simple pole at $a$ with residue $\mathrm{Res}(f'/f,a) = n = \mathrm{ord}_f(a)$. Notice that when $f(a) \in \mathbb{C}^*$, $f'/f$ has no pole at $a$ but it can have a zero when $f'(a) = 0$.
$\zeta$ has exaclty one pole, at $a = 1$, and it is simple thus $\mathrm{Res}(\zeta'/\zeta,1) = -1$ (and not $1$ as you wrote). It also have an infinity of zeroes but none one them one the real half-line $[0,+\infty[$ (I guess that you are interseted by the $s > 1$ so the question of the zeroes doesn't matter).
That is false. The zeta function is analytic with a nonzero value there: $\zeta(0) = -1/2$. And $\zeta'(s)/\zeta(s)$ at $s=1$ has residue $-1$, not $1$.
Quite generally, when $f(s)$ is a meromorphic function, the poles of $f'(s)/f(s)$ are at the zeros or poles of $f(s)$, and they're all simple poles of $f'(s)/f(s)$ with ${\rm Res}_{s=a}(f'/f)$ being equal to the order of vanishing of $f(s)$ at $s = a$ (negative at poles, positive at zeros): $$ f(s) = c(s-a)^m + {\rm higher \ order \ terms} \Longrightarrow \frac{f'(s)}{f(s)} = \frac{m}{s-a} + {\rm higher \ order \ terms}. $$ where $c \not= 0$.
Example. The function $\zeta(s)$ has a simple pole at $s = 1$, so $\zeta'(s)/\zeta(s)$ has a simple pole at $s = 1$ with residue $-1$. Thus $-\zeta'(s)/\zeta(s)$ at $s = 1$ has residue $1$; maybe that is what you meant to write in your question (a sign error).
The formula you wrote down for $-\zeta'(s)/\zeta(s)$ is only valid when ${\rm Re}(s) > 1$, but the zeros and poles of $\zeta(s)$ are all outside that half-plane, so you can't use that formula to study $\zeta(s)$ near $s = 0$, or really anywhere outside of ${\rm Re}(s) > 1$.