Given $A \in \mathbb{R}^{m x n}$, $b \in \mathbb{R}^m$ and $\alpha \in \mathbb{R}_+$
$ \min_{x \in \mathbb{R}^n} $ $f(x) = \frac \alpha2 ||x||_2^2 - \sum^m_{i=1} log(g_i(x)) $
where $g(x) = b - Ax$ and $g_i(x)$ is the i-th component of $g(x)$
Write the gradient vector and Hessian of $f$
I am pretty new to this an I don't really know how to start
How to calculate the gradient vector:
First I decompose $f(\mathbf{x}) = f_1(\mathbf{x}) = f_2(\mathbf{x})$ where $$ f_1(\mathbf{x}) = \frac{\alpha}{2}\left\lVert \mathbf{x} \right\rVert_2^2 $$ and $$ f_2(\mathbf{x}) = - \sum_{i=1}^m h_i(\mathbf{x}) = -\sum_{i=1}^m \log (g_i(\mathbf{x})). $$ For the first term the gradient is very nice $$ \nabla f_1(\mathbf{x}) = \alpha \mathbf{x} $$ and for the second we can write $$ \nabla f_2(\mathbf{x}) = -\sum_{i=1}^m \nabla h_i(\mathbf{x}) $$ where $h_i(\mathbf{x}) = \log ([b - A x]_i) = \log (b_i - A_i x)$. We assume that $b_i > A_i x$ (if not our function is not defined) and note that $$ \nabla (b_i - A_i x) = A_i $$ Further recall that $(\log x)' = \frac{1}{x}$. Using this we can make use of the chain rule $$ \nabla h_i(\mathbf{x}) = \frac{1}{g_i(\mathbf{x})} \cdot \nabla g_i(\mathbf{x}) = \frac{1}{g_i(\mathbf{x})} \cdot A_{i} $$ where $A_{i}$ is the i'th row of $A$. Inserting everything back into our original function we arrive at $$ \nabla f(\mathbf{x}) = \alpha \mathbf{x} - \sum_{i=1}^m \frac{A_i}{g_i(\mathbf{x})} $$ Now this has perhaps given you a taste of the tools you need to use in such calculations and I am hoping you can use these tools to calculate the Hessian on your own.