Ideas for Reducing Problem Dimension in Optimization Problem

Question

Suppose I have a "black box" function $f: \mathbb{U} \rightarrow \mathbb{R}$ where $\mathbb{U} \subset \mathbb{R}^6$and I start with an initial guess $x \in \mathbb{U} = (a,b,c,d,e,f). $ I have already written an optimization algorithm that finds the global optimum of this problem. However, I've noticed that usually one or more of the elements in the initial guess (so either a, b, c...) remains unchanged from its value in the initial guess to its value in the global optimum. Essentially this means that I could have run the optimization algorithm at a lower dimension than 6 (which would save enormous amounts of time). My question is how can I determine which variables are essentially irrelevant before I run my algorithm? NOTE: Just because a variable remains unchanged does not mean it has no effect on the value of the function. It just doesn't change from the initial guess to the optimum.

Answer 1

I am not sure if this will be of any interest to you but I give you here some experience I had with large optimizations of chemical plants (hundreds of variables) using simulation.

I built an elimination procedure based on the derivative analysis. The variables I used to eliminate were those which derivatives were the smallest during the base case (first cycle of the optimizer). But, what I also noticed is that this elimination procedure was to be used in conjunction with Broyden; otherwise, equality or inequality specifications may not be possible.