If $\sum_j \overline{E_j AE_j^\dagger}=\sum_j E_j AE_j^\dagger$ for all $A$ what are the conditions on the sequence $E_j$?

Question

I ran into a programming delemma where I have a system of matrix equations and need to determine exact conditions on the matrices. Let $E_j$ be a finite sequence of $n \times n$ complex matrices such that : $\sum_j E_j E_j^\dagger=I.$ and $j\leq n^2$. Suppose $$\sum_j \overline{E_j AE_j^\dagger}=\sum_j E_j AE_j^\dagger$$ for all symmetric positive semi-definite real matrices $A$ with $trA=1$. Here the bar denotes the complex conjugate and the dagger is the adjoint. Is there any way to classify the conditions on the matrices $E_j$? I know that for example, if we assert $\overline{E_j AE_j^\dagger}=E_j AE_j^\dagger$ for all $j$ we get that each $E_j$ must be real when multiplied by some phase $e^{i \theta_j}$. There are also cases where $$\overline{E_j AE_j^\dagger} \neq E_j AE_j^\dagger $$ for all $j$ and the conditions are still satisfied for all $A$. I was hoping someone could tell me the general conditions required on the matrices $E_j$.