Prove the reduced Riesz representation theorem for finite-dimensional vector spaces with using only the concepts in linear algebra

Question

If someone did not study functional analysis but just studied linear algebra, how to let them understand the idea of Riesz representation theorem for finite-dimensional vector spaces?

The Riesz representation theorem Wikipedia

Let $H$ be a Hilbert space, and let $H^*$ denote its dual space, consisting of all continuous linear functionals from $H$ into the field $\mathbb{R}$ or $\mathbb{C}$. If $x$ is an element of $H$, then the function $\varphi_{x}$, for all $y$ in $H$ defined by: \begin{align*} \varphi_x (y) = \langle y,x \rangle \end{align*} where $\langle \cdot,\cdot \rangle$ denotes the inner product of the Hilbert space, is an element of $H^*$. The Riesz representation theorem states that every element of $H^*$ can be written uniquely in this form.

This description is abstract to me. Since linear algebra is sort of the reduced functional analysis, at the very first step, I am thinking to understand the reduced Riesz representation theorem applied to linear algebra.

In linear algebra, we intend to solve the problem of a linear system \begin{align*} A x = b \end{align*} where $A \in \mathbb{R}^m \times \mathbb{R}^n$ is an $m$ by $n$ matrix, $x \in \mathbb{R}^n$ is an $n$ by $1$ column vector and $b \in \mathbb{R}^m$ is an $m$ by $1$ column vector. The matrix $A$ transforms vectors in $\mathbb{R}^n$ to vectors in $\mathbb{R}^m$, thus we say $A: \mathbb{R}^n \to \mathbb{R}^m$. But the vector $b$ is actually in the column space of $A$, say $C(A) = \mathbb{R}^r \subset \mathbb{R}^m$, which has dimension $r$ that denotes the rank of $A$. Thus we can say $A: \mathbb{R}^n \to \mathbb{R}^r$. If we have an $m$ by $1$ column vector $y$, then we can write \begin{align} y^T A x = y^T b \end{align} We can rewrite it in the form of inner product \begin{align} \langle y,Ax \rangle = \langle y,b \rangle \end{align} And if we consider $b$ as a functional in the dual space of $\mathbb{R}^r$, denoted by $\varphi_{Ax}(\cdot) := \langle \cdot,b \rangle$, then \begin{align} \varphi_{Ax} (y) = \langle y, A x \rangle \end{align} Note that the mapping between $b$ and $\varphi_{Ax}$ is one-to-one. We say, every $b$ in $\mathbb{R}^r$ can be written uniquely in this form. It is very close to the equation in the Riesz representation theorem, but it seems we have to use $Ax$ instead of $x$, unless $A=I$ and $m=n=r$?

I am trying to state the reduced version of the Riesz representation theorem in linear algebra, as follows:

$\mathbb{R}^r$ is a Hilbert space, and its dual space $(\mathbb{R}^r)^*=\mathbb{R}^r$, consisting of all continuous linear functionals from $\mathbb{R}^r$ into the field $\mathbb{R}$. If $Ax$ is an element of $\mathbb{R}^r$, then the function $\varphi_{Ax}$, for all $y$ in $\mathbb{R}^r$ defined by: \begin{align*} \varphi_{Ax} (y) = \langle y,Ax \rangle \end{align*} where $\langle \cdot,\cdot \rangle$ denotes the inner product of the Hilbert space, is an element of $(\mathbb{R}^r)^*$. The Riesz representation theorem states that every element of $(\mathbb{R}^r)^*$ can be written uniquely in this form. That is, every vector $b$ in $\mathbb{R}^r$ can be represented by $\langle y,Ax \rangle$.

This looks like a connection to the "weak formulation" of $Ax = b$, namely, we can find the solution $x \in \mathbb{R}^n$ of $Ax = b$, if for every "test" vector $y \in \mathbb{R}^m$ there holds $\varphi_{Ax} (y) = \langle y,Ax \rangle$.

I am still not fully understand the theorem at this moment, so there might be something wrong stated above. Any comments? Could you provide me with a more clear structure of the reduced Riesz representation theorem in linear algebra?

In addition, the proof of the Riesz representation theorem in textbooks usually take with a nullspace of $\varphi$ denoted by $\mathrm{ker}(\varphi)$ and its orthogonal space $\mathrm{ker}(\varphi)^{\perp}$. In linear algebra, we know the row space of a matrix is always orthogonal to its nullspace. Is there any connection between these two ideas? In other words, can we prove the reduced Riesz representation theorem for finite-dimensional vector spaces with using only the concepts in linear algebra?

Answer 1

Here is a formulation of Riesz representation theorem from mine lecture notes which you might find helpfull.

Let $V,W$ be vector spaces over a field $K$ with $\dim(V), \dim(W)< \infty$ and $f:V\times W \rightarrow K$ be a nondegenerate bilinear form. Then for every $\pi \in W^*$there exists $v \in V$ such that $$\pi=f(v,\cdot)$$

Answer 2

Every non-trivial linear functional $\Phi$ on a vector space $V$ is characterized by its null space $N=\mathcal{N}(\Phi)$, which is always of co-dimension one in $V$. This is because $\Phi(v)=1$ for some $v$, which allows you to write $$ w = (w-\Phi(w)v)+\Phi(w)v, $$ and $w-\Phi(w)v\in N$ is easily directly verified. So $V=N\oplus[\{v\}]$. Conversely, every such decomposition defines a unique linear functional $\Phi$ such that $N=\mathcal{N}(\Phi)$ and $\Phi(v)=1$. To see that $\Phi$ is unique, suppose $\Psi$ is another such linear function. Then $\Phi(w-\Phi(w)v)=0$ implies $\Psi(w-\Phi(w)v)=0$ or $\Psi(w)=\Psi(v)\Phi(w)=\Phi(w)$ for all $w$. This is true for finite- or infinite-dimensional real or complex linear spaces.

If $\Phi$ is a non-zero continuous linear functional on a Hilbert space $V$, then $N=\mathcal{N}(\Phi)=\Phi^{-1}\{0\}$ is closed and, therefore, there is a unique vector $v$ such that $v \perp N$. So $V=N\oplus [\{v\}]$, where the decomposition is orthogonal. And we can assume $\Phi(v)=1$ by multiplying $v$ by an appropriate scalar. The linear functional $\Psi(w)=\frac{\langle w,v\rangle}{\langle v,v\rangle}$ has the same null space as $\Phi$ and $\Psi(v)=\Phi(v)=1$. Therefore $\Psi=\Phi$.

A Hilbert space is needed in infinite dimensions in order to come up with a vector that is orthogonal to the null space of a given continuous linear functional. Continuity ensures that the null space is closed, so that this may be done. In finite-dimensional spaces, only Gram-Schmidt is needed.