Let $A\in\mathrm{GL}_n(\mathbb{C})$, and let's consider its decompositions into a sum of rank 1 matrices $$A=\sum_{i=1}^t A_i,\ \text{rank}(A_i)=1,$$
where $t\geqslant n$. When $t=n$, Wikipedia claims that if we write $A_i=u_iv_i^T$ for some vectors $u_i,v_i\in \mathbb{C}^n$ and collect all the vectors into matrices $U=[u_i]_{i=1}^n,\ V=[u_i]_{i=1}^n$, then for a general matrix $X\in\mathrm{GL}(\mathbb{C})$ defining $c_i,d_i\in \mathbb{C}^n$ by $$UX^{-1}=[c_i]_{i=1}^n,\ VX^{T}=[d_i]_{i=1}^n,$$ yields a new decomposition. It refers to Joe Harris' "Algebraic Geometry: A First Course", but I couldn't find this result there. It's also not immediately clear why acting by $X$ like this on a decomposition yields all decompositions.
But I am also interested in the more general question when $t>n$, especially when furthermore $t\geqslant 2n-1$, which is the dimension of the space of rank 1 matrices. What can be said about such decompositions?
First of all, let's show for good measure that given such matrices $U$ and $V$, the matrices $UX^{-1}$ and $VX^T$ (with $X \in GL_t$) yield a valid decomposition. Note that the columns of $U,V$ correspond to a decomposition of $A$ if and only if $UV^T = A$. Thus, it suffices to note that for any $X \in GL_t$, we have $$ (UX^{-1})(VX^T)^T = UX^{-1}XV^T = UV^T = A. $$ Now, we wish to show that all such decompositions can be attained in this fashion (i.e. the action of $GL_t$ is transitive). First, note that it suffices to consider the case where $A = I$ (where $I$ denotes the identity matrix): $A = UV^T$ holds iff $I = (A^{-1}U)V^T$ holds.
So, we are considering all pairs of matrices $U,V \in \Bbb C^{n \times t}$ for which $UV^T = I$. First, we consider only the pairs for which we have $$ U = U_0 := \pmatrix{I_n & 0}. $$ Verify that an $n \times t$ matrix $V$ will satisfy $U_0V^T = I$ if and only if $V = \pmatrix{I & W}$, where the matrix $W$ of size $n \times (t-n)$ may be freely chosen.
Proof of Claim: It suffices to show that an arbitrary pair $U_0,V$ lies within the orbit of $U_0,U_0$. As discussed above, it must hold that $V$ has the form $V = \pmatrix{I_n & W}$. Let $X$ be the matrix $$ X = \pmatrix{I_n & 0\\-W^T & I_{t-n}}. $$ Verify that this $X$ satisfies $U_0 X^{-1} = U_0$ and $V X^T = U_0$. $\square$
With the claim proved, it suffices to note that for any pair $U,V$ with $UV^T = I_n$, there exist a matrix $X \in GL_t$ such that $UX^{-1} = U_0$. To that end, it suffices to note that $U$ must have linearly independent rows (since it has full row-rank $n$), which means there exists a matrix $X \in GL_t$ whose first $n$ rows are the matrix $U$, which means that this matrix will satisfy $U_0X = U$, and hence $U_0 = UX^{-1}$.
Regarding the nature of these decompositions, I suspect that it might be helpful to take an orbit-stabilizer kind of approach. In particular, I suspect that the dimension of the variety of such pairs will be the total dimension of the variety $GL_t$ (namely, $t^2$) minus the dimension of the subgroup of stabilizers of any particular pair $U,V$.
To understand the subgroup of stablizers, it suffices to look at the single pair where $U = V = U_0$, with $U_0$ as defined above. Note that $U_0X^{-1} = U_0$ holds if and only if $X$ has the form $$ X = \pmatrix{I_n & 0\\X_{21} & X_{22}}. $$ On the other hand, such a matrix $X$ will satisfy $U_0X^T = U_0$ if and only if it further satisfies $$ X = \pmatrix{I_n & 0\\0 & X_{22}}. $$ So, the stablizer subgroup is isomorphic to $GL_{t-n}$, which has dimension $(t-n)^2$. With that, we conclude that the dimension of the variety of pairs $U,V$ satisfying $UV^T = I$ will be $$ t^2 - (t-n)^2 = n(2t-n). $$ By the correspondence described above, the same should hold for the set of pairs satisfying $UV^T = A$ for any element $A \in GL_n$.