A referee has pointed out to me that it is "well known that a Toeplitz operator is compact if and only if it has finite rank" and pointed me to
R. Douglas: Banach algebra techniques in the operator theory, Academic Press, New York and London, 1972.
I have casually read through the book but I could not find this result as an explicit statement.
Is there another good reference or an accessible reasoning for this fact?
The explicit statement is in remark 7.15, page 182 (not sure if there is a single edition).
Corollary 7.13 implies that $\|T_\varphi\|=\inf\{\|T_\varphi+K\|:\ K\ \text{ compact }\}$. So $$ \|T_\varphi\|\leq\|T_\varphi+K\| $$ for any compact $K$. If $T_\varphi$ is compact, you can take $K=-T_\varphi$ to obtain $T_\varphi=0$.
In other words, the only compact Toeplitz operator is the zero operator.