If we consider the tangent vector field $f: \mathbb{S}^1 \to \mathbb{S}^1$ given by $(x,y) \mapsto (-y,x)$ why is it that the derivative of this vector field points in the center of the circle and is perpendicular to $f$?
I don't think that it is in general true that if you have a tangent vector field, then the derivative of the vector field is perpendicular to it.
It has been a while since I did multivariable calc and there was probably some theorem about conservative vector fields or something that is in play here, but I don' remember it.
Having $f\cdot f' = 0$ is equivalent to having $(f\cdot f)' = 0$. That is, the magnitude of $f$ is constant. Here, $f$ is essentially the tangential velocity of uniform circular motion, which is indeed constant. The derivative pointing toward the center is because the field has to constantly turn toward the center to stay on the circle.