Given two fixed matrices $A\in\mathbb{R}^{n\times m}$ and $B\in\mathbb{R}^{n\times p}$ I have a linear map $F:\mathbb{R}^{pm\times 1}\to \mathbb{R}^{n\times 1}$ that can be described as follows: $$ F(x) = \texttt{sum(} A \odot B \texttt{ reshape(}x\texttt{, (p, m))} \texttt{, axis=1)} $$ where $\odot$ is the elementwise product. Basically, we first reshape $x$ into a matrix $X\in\mathbb{R}^{p\times m}$, then we compute $A \odot (BX)$ which is a $n\times m$ matrix, and finally we sum across the columns to get a $n$-dimensional vector.
What is the adjoint operator?
I would like to be able to compute/describe (using math or computer science functions) the adjoint of this map. How can I do that? I know it will have to be $G:\mathbb{R}^{n\times 1}\to\mathbb{R}^{pm\times 1}$ but I am unsure about how to find it.
Possible Solution
I have been told that the solution looks something like this for $y\in\mathbb{R}^{n\times 1}$ $$ G(y) = \text{vec}(A^\top (y \ominus B)) \in\mathbb{R}^{pm\times 1} $$
where $y\ominus B$ is the product of each element of $y$ with the corresponding row of $B$, and it is a $n\times p$ matrix. $$ y \ominus B = \begin{pmatrix} y_1b_{11} & \cdots & y_1b_{1p} \\ \vdots & \vdots & \vdots \\ y_n b_{n1} & \cdots & y_n b_{np} \end{pmatrix} $$
Following comments
Following the comments the function $F:\mathbb{R}^{pm\times 1}\to\mathbb{R}^{n\times 1}$ can be decomposed into the following operations:
- Reshape Operation: $F_1:\mathbb{R}^{pm\times 1}\to \mathbb{R}^{p\times m}$ with $F_1(x) = \texttt{reshape(}x, \texttt{(p,m))} = X$
- Matrix Multiplication $F_2:\mathbb{R}^{p\times m} \to \mathbb{R}^{n\times m}$ with $F_2(X) = BX$
- Elementwise product: $F_3:\mathbb{R}^{n\times m}\to\mathbb{R}^{n\times m}$ with $F_3(X)= A\odot X$
- Reduce sum $F_4:\mathbb{R}^{n\times m}\to\mathbb{R}^{n\times 1}$ with $F_4(X) = X1_{m\times 1}$
The adjoint of $F$ will be $$ F^* = (F_4 \circ F_3 \circ F_2 \circ F_1) = F_1^* \circ F_2^* \circ F_3^* \circ F_4^* $$ Now the adjoint of the reshape operation is the vectorization operation. The adjoint of $F_2(X) = BX$ is simply $F_2^*(X) = B^\top X$. Luckily $A \odot X$ is self-adjoint.