I'm trying to understand some text in the paper below (section 2. beginning of page 3). The text states that <·> denotes the standard trace inner product, but I can't seem to determine what the "standard trace inner product" is of these two matrices which are not necessarily square.
Yuan, Xiaoming; Yang, Junfeng, Sparse and low-rank matrix decomposition via alternating direction method, Pac. J. Optim. 9, No. 1, 167-180 (2013). ZBL1269.90061.
You set $<A, B> = \operatorname{tr} (A^*B) \textrm { or }\operatorname{tr}({}^t AB)$ whether your matrices have complex or real coefficient (the conjugate gives the transposition in the real case) and it is well definied as soon as $A^*B \textrm { or }{}^t AB$ is square.