I'm trying to find the coefficients of $x$ from $f(x)=( (x+1/x)^a+(x-1/x)^a)^b$ where $a,b\in\mathbb{Z}^+$. I tried reading through some of Wilf's Generatingfunctionology, but I think I am still having trouble.
$$\begin{align} [x^n]f(x)&=[x^n]\bigg( \big(x+\frac{1}{x}\big)^a + \big(x-\frac{1}{x}\big)^a \bigg)^b \\ &=[x^n]\sum_{i=0}^b \binom{b}{i} \bigg(x+\frac{1}{x}\bigg)^{a(b-i)} \bigg(x-\frac{1}{x}\bigg)^{a i} \\ &=[x^n]\sum_{i=0}^b \binom{b}{i} \bigg[\sum_{j=0}^{a(b-i)} \binom{a(b-i)}{j} x^{a(b-i)-2j }\bigg] \bigg[\sum_{k=0}^{ai} \binom{ai}{k} x^{ai-2k }\bigg] \\ \end{align}$$
So now I feel like the trick is just to figure out the coefficients for the two pieces in square brackets, since that means I could use the following $$\begin{align} [x^n]f(x)&=[x^n] \bigg(g(x)\cdot h(x) \bigg)\\ &=\sum_{p} \bigg([x^{n-p}] g(x) \bigg)\cdot \bigg([x^p] h(x) \bigg)\\ \end{align}$$
However, at this point I'm getting a little unsure of how exactly to pull the coefficients out of $g(x)$ and $h(x)$ above without messing up indices or the $n-p$ relationship. Is it just
$$\begin{align} [x^n]f(x)&=[x^n]\sum_{i=0}^b \binom{b}{i} \bigg[\sum_{j=0}^{a(b-i)} \binom{a(b-i)}{j} x^{a(b-i)-2j }\bigg] \bigg[\sum_{k=0}^{ai} \binom{ai}{k} x^{ai-2k }\bigg] \\ &=\sum_{i=0}^b \binom{b}{i} \sum_p\bigg[[x^{n-p}]\sum_{j=0}^{a(b-i)} \binom{a(b-i)}{j} x^{a(b-i)-2j }\bigg] \bigg[[x^p]\sum_{k=0}^{ai} \binom{ai}{k} x^{ai-2k }\bigg] \\ &=\sum_{i=0}^b \binom{b}{i} \sum_p\bigg[\binom{a(b-i)}{j}\binom{ai}{k} \bigg] \\ \end{align}$$
with $(n-p)=(ab-ai-2j)$ and $p=(ai-2k)$. Which simplifies further by $k=(ai-p)/2$ and $j=(ab-ai-n+p)/2$ giving
$$\begin{align} [x^n]f(x)&=\sum_{i=0}^b \binom{b}{i} \sum_p\bigg[\binom{ab-ai}{(\frac{ab-ai-n+p}{2})}\binom{ai}{(\frac{ai-p}{2})} \bigg] \\ \end{align}$$
I'm not sure which steps I'm doing wrong, but plugging the last equation into Mathematica and punching in some $a,b,n$ values is giving me some numbers often with a $1/\pi$ term which I know is incorrect since $a,b\in\mathbb{Z}^+$ should be giving me integer coefficients for all $[x^n]$. Any hints, tips, or references?
OK, first of all I would advise that you bring out a factor of $x$ from each of your inner brackets, to get $$\left(\left(x+\frac{1}{x}\right)^{a}+\left(x-\frac{1}{x}\right)^{a}\right)^{b}=\left(x^{a}\left(1+\frac{1}{x^{2}}\right)^{a}+x^{a}\left(1-\frac{1}{x^{2}}\right)^{a}\right)^{b}=x^{ab}\left(\left(1+\frac{1}{x^{2}}\right)^{a}+\left(1-\frac{1}{x^{2}}\right)^{a}\right)^{b} $$ Now, we can certainly expand the inner brackets easily, as: $$2^{b}x^{ab}\left(1+\binom{a}{2}x^{-2}+\binom{a}{4}x^{-4}+\ldots\right)^{b}$$ Now count all the ways we can make $n$ by choosing $b$ even numbers less than $a$ to subtract from $ab$.