I thought I knew what I was talking about, now I'm thoroughly confused. My guess is I've completely misunderstood the notation.
I read in a book that the Lie derivative "preserves the type of tensor fields." I've been taking that to mean that if the "tensor field" was a scalar, that the Lie derivative would return a scalar. Right? Wrong?
Proceeding on the assumption that I'm right, I'd like to check my work. $X$ is a vector field. $f$ is a scalar.
The Lie derivative of a scalar field works out to be \begin{equation}\label{defpart} \mathcal{L}_Xf = X^i\frac{\partial f}{\partial x^i} = Xf \end{equation} Which another book says is "simply the derivative of $f$ in the direction of $X$." In the particular case of a scalar field, of course.
Now here's where I'm pretty sure I go completely off the rails.
Consider this vector field. I'm trying to work out the operator portion -- once you have the operator you can act on any operand. \begin{equation} \vec{X} = 7x\hat{x} + 3y\hat{y} \end{equation}
\begin{equation}\label{one} \mathcal{L}_Xf = X^i\frac{\partial f}{\partial x^i} = Xf \end{equation} According to me (ahem!) \begin{equation} \mathcal{L}_Xf = X^i\frac{\partial f}{\partial x^i} = 7x\frac{\partial f}{\partial x} + 3y\frac{\partial f}{\partial y} \end{equation} But that does not match \begin{equation} 7x\hat{x}f + 3y\hat{y}f \end{equation} For one thing, that's a vector, and for another, $f \ne f^\prime$ in general. But that's what the third part of the definition looks like -- to me.
My problem gets worse if I consider polar coordinates. \begin{equation} \vec{X} = 2r\hat{r} + \theta^3\hat{\theta} \end{equation} Above, I essentially said that the Lie derivative was \begin{equation} \mathcal{L}_Xf = \vec{X}\cdot \nabla f \end{equation} But that doesn't match up with the definition of the book. \begin{equation} \vec{X}\cdot \nabla f = 2r\frac{\partial f}{\partial r} + \frac{\theta^3}{r}\frac{\partial f}{\partial \theta} \ne X^i \frac{\partial f}{\partial x^i} = 2r\frac{\partial f}{\partial r} + \theta^3\frac{\partial f}{\partial \theta} \end{equation}
You can see how messed up I am.
The notation $Xf$ is just shorthand for what you write in the first equality with the derivatives. It doesn't mean to imply ordinary multiplication between the vector and f. The first answer you gave in cartesian coordinates was correct (the second is a vector).
As for the polar coordinates, the book is correct; you've forgotten the form of $\nabla$ depends upon the coordinates due to the non trivial metric. Look up grad in cylindrical or polar coordinates for more info - the angular derivative comes with a 1/r (easy to see on dimensional grounds). Or, if you like, when we raise indices for the contraction $X\cdot \nabla$ you need to use the metric.