What is the relation between a matrix as a linear function versus the same matrix as a bilinear function?

Question

Given an $n \times n$ matrix $A$, we can define a linear transformation $T: \mathbb{R}^n \rightarrow \mathbb{R}^n$ by $T(x)=Ax$.

We could also define a bilnear function $T: \mathbb{R}^n \times \mathbb{R}^n \rightarrow \mathbb{R}$ by $T(x,y) = y^TAx$.

Is there a relation between these two uses of the matrix?

Also, we could do the same thing with an $n \times m$ matrix and get a linear function $\mathbb{R}^m \rightarrow \mathbb{R}^n$ and a bilinear function $\mathbb{R}^m \times \mathbb{R}^n \rightarrow \mathbb{R}$. Is the relationship between using $A$ to define a linear function versus a bilinear function the same in this case?

Answer 1

Redefining symbols to avoid ambiguity: $T: \mathbb{R}^n \to \mathbb{R}^n$ is the linear map defined as $T(x) = Ax$ and $S: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ is the bilinear map defined as $S(x, y) = y^T A x$.

---

Constructing bilinear functions from linear functions using inner product

One way to understand $S$ is as composition of $T$ with the standard inner product $\phi: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ defined as $\phi(x, y) = y^T x$, namely

$$ S(x, y) = \phi(T(x), y). $$

This view allows us to notice some properties of $S$ based on properties of $T$ and known properties of $\phi$. For example, since $\phi$ is known to be non-degenerate, $S$ is non-degenerate if and only if $T$ is an isomorphism.

The construction is readily generalized to the $n \times m$ case by composing $T: \mathbb{R}^m \to \mathbb{R}^n$ with $\phi$.

---

Change of basis

For a fixed matrix $A \in \mathbb{R}^{n \times n}$ the construction above yields two functions: a linear function $T_A: \mathbb{R}^n \to \mathbb{R}^n$ and a bilinear function $S_A: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$. The construction takes place in a fixed basis, but the resulting functions $S$ and $T$ are independent of basis, so it is natural to ask how their matrix representation changes under basis transformations.

It easy to see that the matrix representing a linear function transforms differently than the matrix representing a bilinear function. Let $B$ denote an invertible matrix describing a change of basis. Then

$$ T_A = T_{A'} $$

whenever $A' = B^{-1}AB$, i.e. the matrices representing a fixed linear map are similar. On the other hand,

$$ S_A = S_{A^{''}} $$

whenever $A^{''} = B^TAB$, i.e. the matrices representing a fixed bilinear map are congruent.

This shows that care must be taken when using matrix representations of linear and bilinear functions. Even when a linear function $T$ and a bilinear function $S$ are represented by the same matrix in one basis, it does not imply that they are represented by the same matrix in another basis (unless the basis transformation is orthogonal).

Answer 2

Any bilinear form $b\colon \mathbb R^n\times\mathbb R^n\to\mathbb R$ naturally corresponds to a linear map $\Phi_b\colon \mathbb R^n\to(\mathbb R^n)^*$, where $(\mathbb R^n)^*=\operatorname{Hom}(\mathbb R^n,\mathbb R)$ is the dual space. This correspondence is given by \begin{align} b \quad\longmapsto\quad \Phi_b\colon\mathbb R^n&\to(\mathbb R^n)^*,\\ x &\mapsto b(x,-). \end{align} Here $b(x,-)$ denotes the linear map $\mathbb R^n\to\mathbb R$ sending $y$ to $b(x,y)$.

Considering the elements of $\mathbb R^n$ as column vectors, we can consider the elements of $(\mathbb R^n)^*$ as row vectors: for $\rho\in(\mathbb R^n)^*$ and $x=(x_1,\dots,x_n)^T$ we have \begin{align*} \rho(x) &= \rho(x_1 e_1+\cdots+x_n e_n) = x_1 \rho(e_1) +\cdots + x_n \rho(e_n) \\&= \begin{pmatrix} \rho(e_1) & \rho(e_2) & \dots & \rho(e_n)\end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ \vdots \\ x_n\end{pmatrix}. \end{align*} Using this identification, there is an isomorphism ${}^T\colon(\mathbb R^n)^*\to\mathbb R^n$ given by $y\mapsto y^T$.

Now starting with $b(x,y) = y^TAx$ this yields $\Phi_b(x)=(y\mapsto y^TAx)$ which we identified with the row vector $\Phi_b(x)=(Ax)^T$. Using the above isomorphism we then obtain a linear map $\mathbb R^n\to\mathbb R^n$: \begin{align} \mathbb R^n &\xrightarrow{\Phi_b}\ (\mathbb R^n)^*\ \xrightarrow{T} \mathbb R^n,\\ x &\longmapsto (Ax)^T \mapsto Ax. \end{align}

This is how starting with the bilinear map defined by $A$ we obtain the linear map defined by $A$. You can of course go in the other way as well.

---

More abstractly you can think of this in terms of the tensor-hom adjunction together with the (basis dependent!) isomorphism $(\mathbb R^n)^*\cong(\mathbb R^n)$: \begin{align} \{\text{bilinear maps $\mathbb R^n\times\mathbb R^n\to\mathbb R$}\} &\leftrightarrow \operatorname{Hom}(\mathbb R^n\otimes\mathbb R^n,\mathbb R) \\&\cong \operatorname{Hom}(\mathbb R^n,\operatorname{Hom}(\mathbb R^n\to\mathbb R)) \\&= \operatorname{Hom}(\mathbb R^n,(\mathbb R^n)^*) \\&\cong \operatorname{Hom}(\mathbb R^n,\mathbb R^n). \end{align}

Answer 3

Given $$ {\bf y}_{\,1} = \left( {\matrix{ {y_{1,1} } \cr \vdots \cr {y_{h,1} } \cr } } \right) = {\bf A}\,{\bf x}_{\,1} $$ then $$ {\bf Y} = \left( {\matrix{ {y_{1,1} } & \cdots & {y_{1,h} } \cr \vdots & \ddots & \vdots \cr {y_{h,1} } & \cdots & {y_{h,h} } \cr } } \right) = {\bf A}\,{\bf X} $$

Therefore $$ {\bf Y}^{\,T} {\bf Y} = {\bf Y}^{\,T} {\bf A}\,{\bf X} = {\bf X}^{\,T} {\bf A}^{\,T} {\bf A}\,{\bf X} = \left( {\matrix{ {{\bf y}_{\,1} \cdot {\bf y}_{\,1} } & \cdots & {{\bf y}_{\,1} \cdot {\bf y}_{\,h} } \cr \vdots & \ddots & \vdots \cr {{\bf y}_{\,h} \cdot {\bf y}_{\,1} } & \cdots & {{\bf y}_{\,h} \cdot {\bf y}_{\,h} } \cr } } \right) $$