In Silverman's AEC there is the following paragraph. 
Firstly, $F(z_1,z_2)$ lies in $\mathbb{Z}[a_1,...,a_6][[z_1,z_2]]$. Should I treat the $a_i$s as indeterminates or should I treat them lying in some fixed field?
In the latter case, should I only consider the complete local field case, like $\mathbb{Q}_p$?
I don't see how associativity (or the others) holds formally in the power series ring just because we know $(P + Q) + R = P + (Q+R)$ holds for our elliptic curve E. It somehow feels like I should evaluate the $z_i$ but even if for every legal value of $(z_1,z_2)$ holds I don't see why it should hold as an identity.
I haven’t thought about this matter in precisely this way, but let me make a stab at an answer. My approach is irredeemably computational rather than abstract, and I hope you don’t mind. Feel free to convert to modern language.
You may certainly treat the coefficients $a_i$ as indeterminates. Why don’t you simply deal with the addition on the elliptic curve $E$ as applying to points with values in $Z[(a_i)][[x,y,z]]$? You start with $$ Y^2Z+a_1XYZ+a_3YZ^2=X^3+a_2X^2Z+a_4XZ^2+a_6Z^3\,, $$ dehomogenize by setting $Y=1$ to get $$ Z=X^3-a_1XZ+a_2X^2Z-a_3Z^2+a_4XZ^2+a_6Z^3\,. $$ Your next step is to expand $Z$ as a power series in $X$, since you’re looking at the neighborhood of the identity $\Bbb O=(0,1,0)$, where this $X$ is a uniformizing parameter. You get: $$ Z=g(X)=X^3 - a_1X^4 + (a_2 + a_1^2)X^5 + (-a_3 - 2a_1a_2 - a_1^3)X^6 \\+ (a_4 + 3a_1a_3 + a_2^2 + 3a_1^2a_2 + a_1^4)X^7+\cdots\in\Bbb Z[(a_i)][[X]]\,. $$ The result is that $(x,g(x))$ is a point on the dehomogenized curve, and you should think of the formal group $F(x,y)$ as what shows up in the equality $$ \bigl(x,g(x)\bigr)+_E\bigl(y,g(y)\bigr)=\bigl(F(x,y),g(F(x,y))\bigr) $$ by use of the chord process (no tangent, of course!). Then when you add $(x,g(x))$, $(y,g(y))$, and $(z,g(z))$ in both ways, you get each time a point with coordinates in $Z[(a_i)][[x,y,z]]$, each equal to the other from the associativity of addition on $E$. For my money, that’s it.