When to break a matrix into low rank and diagonal component?

Question

I want to decompose $Y \in \mathbb{R^{n \times n}}$ into low rank components such that $$Y = AB^T$$ where $A \in \mathbb{R^{n \times k}}$, $B \in \mathbb{R^{n \times k}}$ and $k < n$. $Y$ is a sparse matrix with non-zero diagonal elements, is it a good idea to decompose $Y$ into just low rank components or low rank components plus a diagonal term? If I don't include a diagonal term then the there is no meaning of decomposing into low rank components because $Y$ is full rank. Am I thinking in the right direction?