The standard inner product between matrices is often chosen to be \begin{align} \langle A,B\rangle=\mathrm{tr}(AB^\intercal)\,. \end{align} I would like to define another product that looks for $3\times 3$-matrices like the following. We define the operator $\hat{\alpha}$ (where $\alpha$ is a positive real number) acting on matrices, such that \begin{align} A=\left(\begin{array}{ccc} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33} \end{array} \right)\,,\qquad \hat{\alpha}(A)=\left(\begin{array}{ccc} a_{11} & \alpha a_{12} &\alpha^2 a_{13}\\ \alpha a_{21} & a_{22} & \alpha a_{23}\\ \alpha^2 a_{31} & \alpha a_{32} & a_{33} \end{array}\right) \end{align} and define the new inner product as \begin{align} \langle A,B\rangle=\mathrm{tr}\left(\hat{\alpha}(A)\hat{\alpha(B)}^\intercal\right)\,. \end{align} Is there another elegant way to express this product? In general, can I represent any inner product between matrices in a similar way to the Frobenius product with traces?
In other words, my inner product is such that the matrices $M(x,y)_{ij}=\delta_{xi}\delta_{yj}$ form an orthonormal basis with norms $\lVert M(x,y)\rVert=\alpha^{|x-y|}$.
I think that your way to expres is elegant enough, but you could use, if you want, the property
$$ tr(\hat \alpha(A) \hat\alpha(B)^T) = tr(\hat\alpha(\hat\alpha(A))B^T)=tr(A \hat\alpha(\hat\alpha(B))^T) $$