minimize a function using SGD

Question

I have to solve the following exercice :

$$ min ||Ax - y||_2 + ||x||_2^2$$with respect to x, where A ∈ $R^{q×p}$, x ∈ $R^p$ and y ∈ $R^q$. Use stochastic gradient descent.

My first question is how do you solve that if q and p have different values.

the function is is convex so we can use SGD. We have : $$\frac{\partial f(x,y)}{\partial x} = A - 2x$$

and SGD goes:
for nb of iterations: $$x_{k+1} = x_k - learningRate × \frac{\partial f(x,y)}{\partial x} $$

Is that correct? How can I implement a simulation of this problem? After creating x, y vectors and Matrix A with the correct dimensions and initialise them with random values, I would need a loss function (like MSE). How do I define it?
Thanks!

Answer 1

If the problem were instead

$$ \min_x ||Ax - y||_2^2 + ||x||_2^2 $$

we will then have

$$ \min_x x^t (A^t A + I)x -2 y^t A x + y^ty $$

so after derivation regarding $x$

$$ 2x^t(A^t A+I)-2y^t A = 0 $$

and solving

$$ x^t = y^t A (A^t A + I)^{-1} $$