Let
- $\Theta$ be a closed $d$-manifold (and hence a metric space with Borel-$\sigma$-algebra $\Sigma \subset 2^\Theta$),
- $S^d$ the set of real symmetric $d \times d$ matrices, which we can identify with the vector space $\mathbb{R}^{\frac{d(d + 1)}{2}}$,
- $M(\Theta; S^d)$ be space of Borel vector measures on $(\Theta, \Sigma)$ with values in $S^d$,
- $C(\Theta; S^d)$ the space of continuous functions from $\Theta$ to $S^d$. (If I should instead consider compactly supported continuous functions, then this is fine, too.)
I am trying to find a duality statement along the lines of $[M(\Theta; S^d)]^* \cong C(\Theta; S^d)$, where $^*$ denotes the dual vector space.
For $d = 1$ I think I know how a isometric isomorphism $\Psi \colon [M(\Theta; S^d)]^* \to C(\Theta; S^d)$ and its inverse looks like, but I am not sure how to generalise this result and its proof to higher dimensions. It should be relatively straightforward since $S^d$ is finite-dimensional, so we should be able to "copy the one-dimensional procedure componentwise", see e.g. the comment on this highly related question or this pretty similar question, but I don't know how to do that.
For $d = 1$ I have the following: consider $$ \Psi: [M(\Theta, \mathbb R)]^* \to C(\Theta; \mathbb R), \qquad Z \mapsto \big(\theta \mapsto Z(\delta_{\theta})\big), $$ where $\delta_{\theta}(A) := \begin{cases} 1, & \text{if } \theta \in A, \\ 0, &\text{else.}\end{cases}$ is the Dirac measure at $\theta \in \Theta$ for $A \in \Sigma$.
This map is clearly linear. It is also injective: for $Z_1, Z_2 \in [M(\Theta; \mathbb R)]^*$ we have $\Psi(Z_1) = \Psi(Z_2)$ if and only if $Z_1(\delta_{\theta}) = Z_2(\delta_{\theta})$ for all $\theta \in \Theta$. By the linearity of $Z_1$ and $Z_2$ this implies that $Z_1$ agrees with $Z_2$ on any spike train $\sum_{k = 1}^{n} a_k \delta_{\tilde{\theta}_k}$ for $n \in \mathbb N$, $(a_k)_{k = 1}^{n} \subset \mathbb R$ and $(\tilde{\theta}_k)_{k = 1}^{n} \subset \Theta$. By the continuity of $Z_1$ and $Z_2$ and the density of those spike trains in $M(\Theta; \mathbb R)$, we can conclude that $Z_1 = Z_2$.
As inverse I suggest $$ \Psi^{-1}(f) := \left(\mu \mapsto \int_{\Theta} f(\theta) \; \text{d}\mu(\theta) \right). $$ Indeed, for $f \in C(\Theta; \mathbb R)$ we have $$ \Psi(\Psi^{-1}(f)) = \Psi\left(\mu \mapsto \int_{\Theta} f(\theta) \; \text{d}\mu(\theta) \right) = \left( \tilde{\theta} \mapsto \int_{\Theta} f(\theta) \; \text{d}\delta_{\tilde{\theta}}(\theta\right) = f. $$ Furthermore, for $Z \in [M(\Theta; \mathbb R)]^*$ we have $$ \Psi^{-1}(\Psi(Z)) = \Psi^{-1}\left(\theta \mapsto Z(\delta_{\theta})\right) = \left( \mu \mapsto \int_{\Theta} Z(\delta_{\theta}) \;\text{d}\mu(\theta) \right). $$ Due to the linearity of $Z$ we have for any spike train as above $$ \big[\Psi^{-1}(\Psi(Z))\big]\left(\sum_{k = 1}^{n} a_k \delta_{\tilde{\theta}_k} \right) = \sum_{k = 1}^{n} a_k Z(\delta_{\tilde{\theta}_k}) = Z \left(\sum_{k = 1}^{n} a_k \delta_{\tilde{\theta}_k} \right). $$ Hence by continuity of $Z$ and the density of the spike trains we conclude $\Psi^{-1}(\Psi(Z)) = Z$.
What are the higher-dimensional analogues of $\Psi$ and $\Psi^{-1}$?
I tried $\Psi(Z) := \big(\theta \mapsto Z(P \delta_{\theta})\big)$, where $P = \text{id}_{d \times d} \in S^d$ and $\Psi^{-1}(f) := \left( \mu \mapsto \text{tr}\left( \int_{\Theta} f(\theta) \; \text{d}\mu(\theta)\right)\right)$, but they don't seem to fulfil either $\Psi \circ \Psi^{-1} = \text{id}_{C(\Theta; S^d)}$ nor $\Psi^{-1} \circ \Psi = \text{id}_{[M(\Theta; S^d)]^*}$.
Update
When equipping $M(\Theta; S^d)$ with the (strong) generalised total variation norm, then by Singer's representation theorem we have the opposite duality $[C(\Theta; S^d)]^* \cong M(\Theta; S^d)$ via the dual pairing $$ \langle \phi, u \rangle_{C(\Theta; S^d) \times M(\Theta; S^d)} = \int_{\Theta} \langle \phi(\theta), u'(\theta) \rangle_{S^d} \; \text{d}| u |(\theta), $$ where $| u | \in M(\Theta; [0, \infty))$ is the total variation measure of $u$ and $u'$ is the Radon-Nikodym derivative of $u$ with respect to $| u |$.
Hence we only have to figure out how $| u |$ and $u'$ look in this specific case. If $u = \sum_{k = 1}^{N} P_k \delta_{\{ \theta_k \}}$, we have $$ | u | = \sum_{k = 1}^{N} \| P_k \|_{S^d} \delta_{\{ \theta_k \}} \qquad \text{and} \qquad u'(\theta_k) = \frac{1}{\| P_k \|_{S^d}} P_k, $$ so that $$ \langle \phi, u \rangle_{C(\Theta; S^d) \times M(\Theta; S^d)} = \sum_{k = 1}^{N} \text{tr}\big(P_k \phi(\theta_k)\big). $$
How about $\Psi^{-1}(f) := (\mu \rightarrow \underline{1}^T \left( \int_{\Theta} d\mu(\theta) f(\theta)d\mu(\theta) \right) \underline{1}$ ?
To recover, we can use something like $d\mu(\theta) = E_{ij,ji} \delta_{\theta}$ where $E_{ij,ji}$ has zeros everywhere except $(i,j)$ and $(j,i)$.
Let me know if this is useful !
Adding based on feedback below: For example if the measure $d \mu(\theta)$ is absolutely continuous w.r.t the Lebesgue measure it can interpreted as $\int M(\theta)^T f(\theta) M(\theta) d\ell(\theta)$ where $\ell(\theta)$ is Lebesgue measure and $M(\theta) d\ell(\theta) = d\mu(\theta)$ where $M(\theta)$ is a $d \times d$ symmetric matrix. We are using the definition of measure and extract $M(\theta)$ and use it to define integral. Probably will work only if we have a fixed reference measure $d\ell(\theta)$ and define $M(\theta)$ for every measure $d\mu(\theta)$ absolutely continuous w.r.t to reference measure. For example, will work for space spanned by set of measures absolutely continuous w.r.t lebesgue measure and the measure derived from the limits of these measures (limits are the dirac delta which u expect.)
you have to use $M(\theta) = E_{ij,ji}$. So u get $\Psi^{-1}(f):$ Using $(E_{ij,ji} \delta_{\theta})$ we get $1^T E_{ij,ji} f(\theta) E_{ij,ji} 1 = 2 ab \times f_{ij} (\theta) + a^2 f_{ii}{\theta} + b^2 f_{jj}(\theta)$. So u can recover the entire $f(\theta)$ this way by varying $ij$ in $E_{ij,ji}$ and the coefficients at $ij,ji$ position which is varying $a,b$.