I have a recursive relationship, where for a given iteration $k$,
\begin{align*} k=0, f(k) &= v_1\\ k=1, f(k) &= v_1 - 2Av_2\\ k=2, f(k) &= v_1 - 3Av_2 + 3ABv_1 - ABAv_2\\ k = 3, f(k) &= v_1 - 4Av_2 + 6ABV_1 - 4ABAv_2 + ABABv_1\\ \end{align*}
where $v_1$ is a $m_1 \times 1$ vector, $v_2$ is a $m_2 \times 1$ vector, $A$ is a $m_1 \times m_2$ matrix, and $B$ is a $m_2 \times m_1$ matrix.
I want to write $f(k)$ as a summation. I can see that the coefficients are the binomial expansion coefficients. Moreover, the coefficients change from 1 to -1. Therefore, I have
$$f(k) = \sum_{i=0}^k \binom{k}{i} (-1)^{i}...$$
to start with. However, I am stumped by the $A$, $AB$, $ABA$, $ABAB$, terms. How can I express those in my summation?
I think the best you can do is $(AB)^{\lfloor i/2 \rfloor} A^{i \text{ mod } 2}$. Though a "definition by example", as you've given above, will likely suffice for the reader to understand what you are defining. You could also define a function such as $m_i = \begin{cases} (AB)^{i/2} & i \text{ even}\\ (AB)^{(i-1)/2}A & i \text{ odd} \end{cases}$. Then you could define your term using this $m_i$.
I hope this helps ^_^