I'm trying to link a machine learning framework to more theoretical considerations about linear isometries.
Let's say we have an input dataset $\mathcal{D}=\{(\textbf{x}_1,y_1), (\textbf{x}_2,y_2), \dots , (\textbf{x}_N,y_N)\}$, $\textbf{x}_i \in \mathbb{R}^d$ and $y_i \in [0,1]$.
Suppose we train a neural network $f_\theta: \mathbb{R}^d \rightarrow \mathbb{R}^m$ to extract representations $\textbf{z}_i$ from original raw data $\textbf{x}_i$ optimizing the following objective:
$$C \cdot\sum_{i=1}^N\sum_{j\neq i}(d(\textbf{z}_i, \textbf{z}_j) - d(y_i, y_j))^2$$
where $\textbf{z} = \frac{f_\theta(\textbf{x})}{\|f_\theta(\textbf{x})\|}$ and $C$ is some normalization constant.
If we then consider the mapping assigning each learnt representation $\textbf{z}_i$ to its corresponding target $y_i$, i.e. $\mathcal{F}: \mathbb{R}^m \rightarrow [0,1]$, can we say that it will be approximately linear? I think it should be an (approximated) isometry given the objective we have optimized, but can we also say something about linearity?