I maneuvered myself into a bit of a corner and would need some help how to proceed or where I've been foolish.
The TL:DR version of my problem is:
I have a functional of the form
$$f: \Xi\to[0,1]~~(\xi)\mapsto g\left(\int_\xi L_1(\vec{r}) ds,\dots, \int_\xi L_N(\vec{r}) ds\right)$$
where $\Xi$ is the the family of (twice differentiable) functions defined on a convex subset $U\subset\mathbb{R}^n$, $$g:\mathbb{R}^n\to [0,1]~~(x_1,\dots,x_n)\mapsto \frac{e^{-x_1}}{\sum_{i}e^{-x_i}}$$
is the first component of the softmax of a vector $\vec{x}$, and $L_i: U\mapsto \mathbb{R}^+$ is a scalar field over $U$. I want to find the path $\xi$ which maximizes the above functional. I've done a bit of variational calculus (mainly variational mechanics), but this doesn't seem to be easily applicable here.
How does one proceed on such a problem?
The long version of my problem is:
Let $U$ be a convex subset of $\mathbb{R}^n$. Suppose there is a finite number of points $g_i \in \mathbb{G}\subset U$ and associated scalar fields $S_i:~U\to \mathbb{R}^+~~(x)\mapsto f(x,g_i,\phi)$ for which an analytical expression of the integral exists, and which are parameterized by some $\phi\in\mathbb{R}^n$ which is shared among all $S_i$. The value of a path $\xi$ in $U$ for each $S_i$ can then be defined via the line integral as
$$C_i(\xi) = \int_\xi S_i(\vec{r}) ds.$$
I want to look at paths $\gamma$ that end in $\mathbb{G}$ and sections $\xi$ thereof, i.e. paths that satisfy $\xi: [0, T_1]\to U ~~ t\mapsto \gamma(t)$ for some $\gamma:[0,T_2]\to U$ that has $\gamma(T_2)\in \mathbb{G}$.
For a section $\xi$ we can define a likelihood of it being associated with with a path that leads to $g_i\in\mathbb{G}$ as corresponding to the value $C_i(\xi)$, with higher values meaning it is less likely to correspond to a path that leads to $g_i$. We can do this, for example, using the softmax
$$\mathrm{softmax}: \mathbb{R}^{|\mathbb{G}|}\to(0,1)^{|\mathbb{G}|}~~(x_1,\dots,x_{|\mathbb{G}|})\mapsto (\frac{e^{x_1}}{\sum_{i}e^{x_i}}, \dots, \frac{e^{x_{|\mathbb{G}|}}}{\sum_{i}e^{x_i}})$$
which leads to a function of the form
$$\Psi_i: \Xi\to[0,1]~~(\xi)\mapsto \vec{\delta_i}\mathrm{softmax}(-C_1(\xi),\dots,-C_N(\xi))$$
where $\vec\delta_i$ is a vector where all components are $0$ except the i-th wich is $1$, i.e. we select the i-th component of the softmax as the probability of the path $\xi$ leading to $g_i$.
I am interested in two problems:
- What is the path $\xi^*_i$ from a given $\xi^*_i(t=0)=u_i\in U$ to $\xi^*_i(t=T)=g_i$ which maximizes the likelihood $f_i$ of leading to $g_i$ along the way?
- Given a (finite) set of pairs $(\xi, \vec{\Psi}(\xi))$, can we estimate $\phi\in\mathbb{R}^N$ (the parameter of the vector fields $C_i$) so that the resulting vector fields most closely describe the set of pairs?
For the first question I am very lost. If somebody has any ideas, I would love to hear them! It is not my question in the title, but I assume this can also be solved by not using the softmax, but some other function to map the path values to something that can be interpreted as a probability.
For the second question, I would use the fact that the softmax is invertible up to a constant and that the integral for $L_i$ has an analytical expression. This should yield a classic machine learning problem.
I find functionals really fascinating, so I would appreciate any help and pointers where to look for more information :)