I want to solve a multivariate optimization problem which is the more complex form of the following: $$\min_{A\in I,X\in J} \|Y-AX\|_F^2+\lambda \|Y-CX\|_F^2,$$ where $(Y, C, A, X)$ are matrices with appropriate sizes, and $(I,J)$ define specific constraints on $(A,X)$, and the scalar $\lambda$ is fixed beforehand.
As a solution, I use the alternating optimization scheme as the loop of: $$\text{1-}\quad X^*=\arg\min_{X} \|Y-A^*X\|_F^2+\lambda \|Y-CX\|_F^2$$ $$\text{2-}\quad A^*=\arg\min_{A} \|Y-AX^*\|_F^2.$$ Although the optimization converges in a limited number of steps, we can see that the second objective ($\|Y-CX\|_F^2$) is eliminated in the optimization step of $A$.
Hence, I was wondering if it'd be more efficient (regarding the convergence) if I re-formulate it as: $$\min_{A,X} \|Y-AX\|_F^2+\lambda \|AX-CX\|_F^2,$$ which makes both objectives to be involved in the optimization loops of both of $(A,B)$: $$\text{1-}\quad X^*=\arg\min_{X} \|Y-A^*X\|_F^2+\lambda \|A^*X-CX\|_F^2$$ $$\text{2-}\quad A^*=\arg\min_{A} \|Y-AX^*\|_F^2+\lambda \|AX^*-CX^*\|_F^2.$$
So, should the second formulation provide any additional benefit compared to the first one regarding the speed and quality of convergence? In that case, how/why?
BTW, the above is a simplified version of the real problem, so any specific solution to this problem does not necessarily applies to the original one.