Can we say anything about the convexity of the $|y-X^Tw|^2$ if we know that inverse of $X$ does not exist?
Hessian is $2XX^T$. Given that X is not invertible, can we conclude that $XX^T$ is indefinite hence $|y-X^Tw|^2$ is non-convex?
y and w are column vectors and X is a matrix. w is a variable.
As the Hessian is positive semidefinite (even if not invertible), the function remains convex (but maybe not strictly convex).
A trivial example is $$ X = \begin{pmatrix} 1 & 0\\ 0 & 0 \end{pmatrix} $$ where $$ |y - X'w|^2 = (y_1 - w_1)^2 + y_2^2 $$ is a convex function, even if it is constant in the direction $\{ w_1 = const. \}$.