Given a multi-variate quadratic function, how can I compute the path of steepest descent to the minimum?

Question

I have a multi-variate quadratic function, $$f(\mathbf{x}) = c + \mathbf{x}^\mathsf{T}\mathbf{g}+\frac12\mathbf{x}^\mathsf{T}\mathbf{H}\mathbf{x} : \mathbb{R}^{|\mathbf{x}|}\rightarrow\mathbb{R},$$ where $\mathbf{H}$ is a real, symmetric, positive definite matrix (such that this function has a minimum). How can I construct a function $$g(\alpha) = \mathbf{x} : \mathbb{R}\rightarrow\mathbb{R}^{|\mathbf{x}|}$$ that outputs the path of steepest descent (i.e. taking infinitesimally small steps, move downhill in the direction of steepest gradient), from $\mathbf{x}=\mathbf{0}$ at $\alpha=0$ to the function minimum ($\mathbf{x}=-\mathbf{H}^{-1}\mathbf{g}$) at $\alpha=1$.

Is it possible to construct this path such that $\alpha$ is proportional to the distance along the path, i.e. $$\frac{d\|g(\alpha)\|}{d\alpha} = k, \forall \alpha \in [0, 1],$$ where $k$ is a fixed?

Many years ago I read a paper explaining how to do the first bit, and I recall (hopefully correctly) that it involved an Eigen-decomposition of $\mathbf{H}$. I can no longer find the paper, so if you have a reference, please share.

Update: Building on the idea of @MatteoDR...

I feel like there is some Lie algebra solution here. The matrix update $$\left[\begin{array}{cc}\mathbf{-H} & \mathbf{-g}\\ \mathbf{0} & 1\end{array}\right]\left[\begin{array}{c}\mathbf{x}\\1\end{array}\right]$$ defines the direction of steepest descent at a given point, $\mathbf{x}$. In essence it defines the manifold over which $\mathbf{x}$ can exist along as it moves downhill. I should be able to recursively apply this update (or the matrix exponential of this update?) to get any position along the trajectory.

Answer 1

Idea: We can compute the solution of the differential equation $\dot x(t)=-\nabla f(x(t))=-g-Hx(t)$ (there is an explicit formula depending on the exponential matrix $e^{-H}$, by variation of the constant principle) starting at $0$. The (trajectory of the) solution starting at $x(0)=0$ is the curve your are looking at. The problem is that, since the convergence to the minimum is exponential, the equilibrium point is reached in infinite time. I.e. the solution will be a parametric definition of your curve of the form $\alpha:[0,+\infty)\to \mathbb{R}^n$ with $\alpha(0)=0$ and $\lim_{t\to \infty} \alpha(t)=-H^{-1}g$. Then, maybe you could reparametrize this curve to obtain a parametrization defined on $[0,1]$ with the required properties.

PS: Maybe one should ask $Q\succ 0$ and not only semidefinite to have uniqueness of minimum.

Answer 2

Building on the idea of @MatteoDR...

I feel like there is some Lie algebra solution here. The matrix update $$\left[\begin{array}{cc}\mathbf{-H} & \mathbf{-g}\\ \mathbf{0} & 1\end{array}\right]\left[\begin{array}{c}\mathbf{x}\\1\end{array}\right]$$ defines the direction of steepest descent at a given point, $\mathbf{x}$. In essence it defines the manifold over which $\mathbf{x}$ can exist along as it moves downhill. I should be able to recursively apply this update (or the matrix exponential of this update?) to get any position along the trajectory. However, I've tried to compute and plot various variations along these lines, and see if the trajectory passes through the minimum, and haven't yet gotten it to work.

My feeling is that I'm close to the solution. It must surely be known, so I'm hoping someone with knowledge in this area will read this post.

I've added this line of thinking to the question.