I am learning linear algebra using the textbook Linear Algebra Done Right, trying to understand the subject through a logical, pure math perspective. I'm, simultaneously, learning applied linear algebra in college, and trying to reconcile the two. Specifically, I am very confused on why dual spaces become used so often in the form of the transpose.
Let me explain. My understanding of the transpose from Linear Algebra Done Right is as follows:
Define on each vector space $V$ another vector space denoted $V^{*}$ which is the space of all linear functional on that space. I.e. it is the space of linear maps from $V$ to $F$, the field of interest.
Intuitively, I understand the dual space of a vector space as the space of all possible weight-assignments to the vectors of a vector space. Each element $f \in V^{*}$ is analogous to a measuring device that assigns particular weights to every vector.
Then by the dual map of a linear map, $T^{*}$, I understand a way of defining a particular measuring device in the domain that agrees with the measures of the co-domain. Let me rephrase this using notation:
Suppose $T: V \rightarrow W$. Then $T^{*}: W^{*} \rightarrow V^{*}$ is defined such that $T^{*}(g)(x) = g(T(x))$.
In other words, $T^{*}$ defines a weight-assignment in $V^{*}$ of the pre-image of $T$ that agrees with the $W^{*}$ weight-assignment of the image of $T$.
...
Now this is all good, but where I become confused in why this would have so many applications. For example:
How should I think of the least-square solution method, where we solve for $T^{*}T(x) = T^{*}(b)$? What is this statement really saying, aside from being a purely computational trick for finding the least square solution. I.e. why does this work on a deeper level?
The projection matrix is given by $A(A^{*}A)^{-1}A^{*}$. Why must this be so, in the context of dual spaces.
More generally, why does the tranpose come up so often as a computational trick to simplify problems? On the face of it, I am having a hard time seeing how the formal definition leads to the practical uses.
It's hard to give an answer to a confused question, so, this is a long comment. Firstly let's use slightly different notation.
Transpose of a linear map $T: V \to W$ is an induced pull-back of linear functionals $g \in W^*$ and let's denote it as $T^t: W^* \to V^*.$
If there is also an inner product on $V$, we can define adjoint of a linear operator $T: V \to V$ as $T^{*}:V \to V$: $\langle T^{*}x,y\rangle=\langle x,Ty\rangle$
Both of your projection-related applications assume inner product and use the adjoint $T^{*}$ (or its matrix $A^{*}$)
One might ask how and why the (matrix representations of) transpose and adjoint are related (can they be equal!?) It has been asked before, for example: Relationship between Definition of Adjoint of a Linear Operator and the Transpose of Matrix
Sorry if this only begins to answer your question(s) by hopefully clearing the confusion