Let $G$ be a compact group, $\alpha$ be a unitary irrep of $G$ with carrier space $\mathcal A$, and $\beta$ be a unitary irrep of $G$ with carrier space $\mathcal B$. Then, the action of $G$ on vectors in $\mathcal A\otimes\mathcal B$ is found by tensoring $\alpha$ with $\beta$, so that
$$(\mathbf{a}\otimes \mathbf{b})\mapsto\alpha_g\otimes\beta_g (\mathbf{a}\otimes \mathbf{b})=(\alpha_g\mathbf{a})\otimes(\beta_g\mathbf{b}),$$
for every $g\in G$, and one can use Clebsch-Gordan coefficients to decompose this tensor representation. What I have, instead, is a representation of $G$ on linear maps $M:\mathcal B\rightarrow\mathcal A$: $$M\mapsto\alpha_g\,M\,\beta_g^\dagger.$$
How can I decompose this representation? I have thought that it might be related to the decomposition of a representation of $G$ acting on $\mathcal A\otimes\mathcal B^*$, so that the action of $G$ can be written as
$$\alpha_g\otimes \beta_g^\dagger(\mathbf{a}_i\otimes\mathbf{b}^\dagger_j)=(\alpha_g\mathbf{a}_i)\otimes (\mathbf{b}^\dagger_j\beta_g^\dagger).$$
I am trying to understand if I can use the standard Clebsch-Gordan coefficients to decompose $\alpha_g\otimes \beta_g^\dagger$ (because $\mathcal B$ and $\mathcal B^*$ are canonically isomorphic, so the solution could be as simple as suitably modifying the C-G coefficients, perhaps with a complex conjugate in the right place, or a transposition). But then I'm thrown off by the fact that I don't really have things like $\alpha_g\otimes \beta_g^\dagger$, but rather things like $\alpha_g\,(\#)\,\beta_g^\dagger$, where $\#$ is a placeholder.
Your representation is just the representation of $G$ on $\mathcal{A} \otimes \mathcal{B}^{*}$, as you noticed. But as $G$-representations, $\mathcal{B}$ and $\mathcal{B}^{*}$ are isomorphic by the map $$b \mapsto (b, - )$$ which is $G$-invariant because, for any $g \in G, b, b'\in \mathcal{B}$, $$g\cdot b (b') \mapsto (g\cdot b, b') = (b, g^{-1} \cdot b) = (g\cdot (b, -)) (b').$$
I think you actually noted the above. So, if $\mathcal{B}$ and $\mathcal{B}^{*}$ are isomorphic as $G$-representations,
$$\mathrm{Hom}_{\mathbb{C}}(B, A) \cong \mathcal{A} \otimes \mathcal{B}^{*} \cong \mathcal{A} \otimes \mathcal{B}$$ so the decomposition into irreducibles should be the same.