In calculus, how should I interpret the -1 superscript in trigonometric functions?

Question

In calculus, and in the context of differentiating functions for practice, how should I interpret the following expression (i.e., what is the convention here) $$f(x) = \tan^{-1}(x)$$ Should I treat it as the inverse of tan or just $$\frac{1}{\tan(x)}$$

Answer 1

Short answer: This is a terrible collision of notation due to the fact that there are two kinds of binary operations on functions that are often denoted by concatenation, but the $-1$ exponent usually denotes the function composition inverse of the trig function that inputs a number and outputs an angle.

---

Longish answer: I have to clarify this every semester in my algebra/calculus classes, so here are some more complete thoughts.

Suppressing domain considerations, given functions $f$ and $g$, we can form the pointwise product whose value at $x$ is $f(x)\,g(x)$, or we can form the composition $\smash{f\bigl(g(x)\bigr)}$, either of which is often denoted by $fg$, depending on context. (Sometimes the composition is denoted $f \circ g$ to explicitly disambiguate the two, but read on.)

Each operation has an identity, say $e$. For multiplication, this is the constant function $x \mapsto 1$, so $fe = ef = f$ for any function $f$ since $f(x) \, 1 = 1 \, f(x) = f(x)$ for all $x$. However, for composition, this is the identity function $x \mapsto x$ so $fe = ef = f$ for any function $f$ since $f(e(x)) = e(f(x)) = f(x)$ for all $x$ in this case.

Going further, we can combine a function with itself in either way. Should $f^2 = ff$ denote the product that evaluates to $\smash{\bigl( f(x) \bigr)^2} = f(x) f(x)$? Or should it denote the composition $\smash{f\bigl( f(x) \bigr)}$? Analogously, for any natural $n$, is $f^n$ the $n$-fold pointwise product $f(x) \cdots f(x)$ or the $n$-fold nested composition $f(f(\cdots f(x) \cdots ))$?

And then there's the inverse, when it exists: The inverse of $f$ is the function $f^{-1}$ such that $ff^{-1} = f^{-1}f = e$, the identity. If you are working with multiplication, then $f^{-1}(x) = \smash{\bigl( f(x) \bigr)^{-1}} = 1/f(x)$. But if you are working with composition, the inverse is the function where $f^{-1}(x) = y$ means that $f(y) = x$.

This is a real problem with the trig functions, where we are genuinely interested in both kinds of inverse! To avoid this ambiguity, many of us avoid the superscript $\tan^{-1}$ notation entirely, instead opting for $$ \cot x = \frac{1}{\tan x} $$ for the multiplicative inverse, and $$ \arctan x $$ for the compositional inverse, since either could reasonably be $\tan^{-1} x$.

---

Addendum 1. This problem becomes even more muddled in linear algebra, where two linear transformations, say $S$ and $T$, are represented in coordinates by matrices, say $A$ and $B$, and the composition $ST$ is represented by the product matrix $AB$, and that operation on the matrices is called multiplication (for good reasons, but I digress).

(See comment by David Z below.) Addendum 2. In the object oriented paradigm in computer science, this multiple use of the same name that depends on context is celebrated under the name polymorphism, where objects of different classes can support a method of the same name.

---

The moral is that context matters, and notation only carries as much information as you imbue it with. Often, context is suppressed by relying on conventions, which is essentially what your question (and other spiritually similar questions about order-of-operations, etc.) but we suppress that context at our peril! We must aspire to communicate mathematics concisely, but unambiguously. Remember: The notation works for you, not the other way around.

Answer 2

$\tan^{-1}x=\arctan x$. It is an unfortunate confluence of conventions, but fortunately the function $1/\tan x$ has a name, $\cot x$, making it easier to disambiguate.