I was reading a book on non-linear multiobjective optimization by Kaisa M. Miettinen and in a paragraph the author says:
"In single objective optimization problems, the main focus is on the decision variable space. In the multiobjective context, we are often more inter- ested in the objective space. For one thing, it is usually of a lower dimension than the decision variable space."
My question is why the dimension of the objective space is usually lower than the dimension of the design space?
I think because if $X \subset \mathbb{R}^n$ is the feasible space and $F:\mathbb{R}^n \to Range($F$) \subset \mathbb{R}^n$ where $F(x)=[f_1(x),f_2(x),..,f_k(x)]^T$ and $x \in \mathbb{R}^n$ and the objective space is $Y=F(X)$. If we know that all $f_i$ $i=1,2,..,k$ are linear then we can prove that $F$ is a linear operator and use the result from functional analysis that say if $F:X \to Range($F$)=Y \subset \mathbb{R}^n$ is a linear operator then:
- Range($F$)=Y is a vector space
- If dom($F$) has dim=n then the Range($F$) has dim $\leq n$
Question What if $f_i$ are non-linear can anything be deduced?
The @littleO comment answers this, here is just some expansion:
If $X \subseteq \mathbb{R}^n$ and $F(X)\subseteq \mathbb{R}^k$ then the book just means that $k$ is typically much smaller than $n$.
An example I often use is this: Imagine a communication system where we allocate power as a vector $x=(x_1, ..., x_n)$ over $n$ different subchannels (such as $n$ disjoint subbands in the frequency spectrum), where $n>100$: $$ X = \left\{(x_1, ..., x_n)\in \mathbb{R}^n : x_i \geq 0\quad \forall i \in \{1, \ldots, n\}\right\}$$ The dimension of the decision set $X$ is larger than 100, but suppose we have only 2 objectives we care about: \begin{align} f(x) &= \sum_{i=1}^n x_i = \mbox{ total power} \\ g(x) &= -\sum_{i=1}^n \log(1+s_ix_i) = \mbox{ $(-1) \times $(sum transmission rate) (bits/sec)} \end{align} were $s_1, ..., s_n$ are given attenuation coefficients for each subchannel, and $\log(1+s_ix_i)$ is the normalized Shannon capacity on channel $i$ associated with allocating $x_i$ units of power. We want to make "both" $f(x)$ and $g(x)$ small.
Now we can define the set $$A = \{(f,g) \in \mathbb{R}^2 : (f,g)=(f(x),g(x)) \quad \mbox{for some $x \in X$}\}$$ So we can view the problem as picking a good point $(f,g)$ in the 2-dimensional set $A$. Optimization over the 2-dimensional set $A$ is easier to geometrically understand, and there are simple concepts of Lagrange multipliers that are useful here. I use this example in my course notes, Theorem II.1 on page 5 gives a very simple but surprisingly useful result for general (possibly discrete and nonconvex) sets $A$ that connects Pareto optimality, constrained optimization, and Lagrange multipliers.
http://ee.usc.edu/stochastic-nets/docs/network-optimization-notes.pdf
On dimensionality of nonlinear maps: It can be shown that for any positive integer $k$, the function $h:\mathbb{R}\rightarrow\mathbb{R}^k$ given by $h(t) = (t, t^2, t^3, ..., t^k)$ is such that the set $h(\mathbb{R})$ (or even the set $h([0,1])$) cannot be contained in any affine subset of $\mathbb{R}^k$ of dimension smaller than $k$.
*"affine subset" = "subspace + constant"