How can I get the principle 2d "slice" (of possibly limited size) of a function $R^N→R$ such that the variance of values inside is maximized?
For example, when trying to optimize the matrix $W$ in order to minimize the loss:
$$loss(W)=||X.W-Y||_2$$
where $X$ and $Y$ are given matrices. The loss is basically a function:
$$loss:R^n→R$$
where $n$ is the number of numbers in $W$. The actual loss function might be more complicated, for example:
$$loss(W,B,V,C)=||tanh(X.W+B).V+C-Y||_2$$
where $tanh$ is element-wise, $W$, $B$ are matrices and $V$, $C$ are column vectors (or matrices with identical rows). Since the problem might be complicated and hard to analyze, I want to have a physical (or graphical) understanding of the loss function.
The problem is, since the dimensionality of the function might be high, it's hard to directly visualize the function. Therefore, similar to PCA (Principle Component Analysis), I'm wondering is the a way to take a 2d (finite?) plane (or "slice") inside the domain of the loss function such that I can see the most "interesting" part of the function (e.g. maximize the variance of the function values inside the plane)
I tried to convert this problem to PCA, but since PCA cares only about position of the observations in the space (rather than their corresponding values), I can't figure out how to do the conversion.
Thank you!