I am reading about the formal multiplicative group, with addition given by $F(x,y)=x+y+xy$, and I am wondering if there is a nice way to describe the inverse of an element.
So if I let $x+y+xy=0$, then I can rearrange to get $y=-x/(1+x)$, however this is not a formal power series. In calculus, we can express the function $1/(1+x)$ as a convergent power series if $|x|<1$, but this doesn't make sense in a formal group. What am I missing? Is there a better interpretation?
First, the whole point of formal power series is that there is no need to consider convergence. These are just formal expressions; they are not functions, and such an expression need not make any sense if we plug in a nonzero value for $x$. To slightly alter a phrase due to Wilf, a formal power series is simply "...a clothesline on which we hang up a sequence of numbers for display."
More rigorously, we can define $R[[x]]$ as the countably infinite direct product $\prod_{i=0}^\infty R$, so as a set, $R[[x]]$ is simply the set of all (countably infinite) sequences $(a_0, a_1, a_2, \ldots)$ with $a_i \in R$ for all $i$. We denote a sequence $(a_0, a_1, a_2, \ldots) \in R[[x]]$ by $$ \sum_{n=0}^\infty a_n = a_0 + a_1 x + a_2 x^2 + \cdots \, , $$ but this is just notation: $x$ is simply a shorthand for the sequence $(0,1,0,0,\ldots)$, and similarly $x^n$ is sequence consisting of all zeroes except for a $1$ in the $n^\text{th}$ spot.
We equip $R$ with pointwise addition and define multiplication by the familiar Cauchy product: if $f = \sum_n a_n x^n$ and $g = \sum_n b_n x^n$, then $fg = \sum_n c_n x^n$, where $$ c_n = \sum_{k=0}^n a_{k} b_{n-k} \, . $$ Using this definition, we can easily see that $\sum_{n=0}^\infty (-1)^n x^n$ is the multiplicative inverse of $1+x$:
\begin{align*} \require{cancel} c_0 &= a_0 b_0 = 1 \cdot 1 = 1\\ c_1 &= a_0 b_1 + a_1 b_0 = 1 \cdot (-1) + 1 \cdot 1 = 0\\ c_n & = \sum_{k=0}^n a_{k} b_{n-k} = a_0 b_n + a_1 b_{n-1} + a_2 b_{n-2} + \cdots + a_n b_0\\ &= 1 \cdot (-1)^n + 1 \cdot (-1)^{n-1} + \cancelto{0}{0 \cdot (-1)^{n-2}} + \cdots + \cancelto{0}{0 \cdot 1} = 0 \end{align*} for $n \geq 2$. Really, this is just a formal way of writing the following calculation that you might have seen in calculus. \begin{array}{cccc} (1+x)(1 - x + x^2 - x^3 + x^4 - \cdots) & = & & 1 &-& x &+& x^2 &-& x^3 &+& x^4 &-& \cdots\\ & & & & + & x & - & x^2 & + & x^3 & - & x^4 & + & \cdots\\[5pt] \hline\\ & & & 1 &+& 0 &+& 0 &+& 0 &+& 0 &+& \cdots \end{array} However, we can meaningfully talk about convergence in $R[[x]]$ by endowing $R[[x]]$ with a natural topology called the $(x)$-adic topology. (See here or here.) This topology arises from the following discrete valuation: given a series $f \in R[[x]]$, write $f(x) = x^m g(x)$ where $g(0) \neq 0$ (i.e., $x \nmid g$). (This can be done because $R[[x]]$ is a UFD---a DVR, in fact.) Define the $(x)$-adic valuation $v(f)$ of $f$ to be $m$, and the $(x)$-adic absolute value of $f$ to be $|f| := 2^{-m}$. One can show that this defines a nonarchimedean absolute value that turns $R[[x]]$ into a topological ring.
This topology is useful because it allows us to work with finite partial sums and then take the limit, as done in calculus. It also allows us to say when we can meaningfully compose two series: when does $f(g(x))$ make sense for $f, g \in R[[x]]$? The key fact is that if something is $0$ mod $x^N$ for all $N$ sufficiently large, then it is $0$. For a toy example, we can revisit the problem of computing $\frac{1}{1+x}$. One can easily show that $$ \lim_{N \to \infty} \sum_{n=0}^N (-1)^n x^n = \sum_{n=0}^\infty (-1)^n x^n \, . $$ Computing with a finite sum, we see that \begin{align*} (1+x)(1 - x + x^2 - x^3 + x^4 - \cdots \pm x^N) = 1 \pm x^{N+1} \equiv 1 \pmod{x^{N+1}} \end{align*} for all $N \geq 0$. Applying $\lim_{N \to \infty}$, we obtain the result as before.
This was a toy example, but the same approach can be used to show that, substituting $y = -x \sum_{n=0}^\infty (-1)^n x^n$ into $F$, the result converges $(x)$-adically to $1$. We need the topology to do this because we are trying to treat $F$ as a function and plug in a value for $y$, which yields a composition of formal power series. I suppose since $F$ is just a polynomial in this example, we don't actually need to invoke the topology, but for other formal group laws where $F$ is truly an infinite series, such as the hyperbolic tangent group law $F(x,y) = \frac{x+y}{1 + xy}$, the topology becomes indispensable.
For more on the topological properties of $R[[x]]$, see these notes (Waybackup).