Saying a matrix is "singular" means that it cannot be inverted. Separately "singular value decomposition" factors a matrix into three matrices, one that contains "left-singular" vectors as columns, one that contains "singular values" on the diagonal, and another that has "right-singular" vectors as columns. And ironically, SVD works on a matrix regardless of whether or not it is singular.
Is there some underlying sense in which singular describes something common to all of these? If I accept that "singular values" was an arbitrary choice, then I can accept that SVD has singular in the name because it uses them, and that left and right singular vectors are just so called because they appear on either side of the matrix that contains singular values, but then I'll still left wondering how we get from there to singular matrices.