In this paper by Kaplansky on his theorem of projective modules over local rings, he states
Hence the matrix $(c_{ij})$ is non-singular; for it has units down the main diagonal and non-units elsewhere, and over a local ring this suffices to make a matrix non-singular.
Why are matrices with units on the diagonal and non-units off the diagonal invertible in a local ring?
The last paragraph of this blog post on the paper discusses one approach but I'm still missing something. It proves that such a matrix is the sum of an invertible matrix and a matrix in the Jacobson radical, but why is that sum invertible? I think it has something to do with "$x$ is in the Jacobson radical of $R$ iff $1-xr$ is a unit for all $r \in R$."
I believe I solved my question. The answer follows the blog post linked in the question.
Write $(c_{ij}) = D + E$ where $D$ is the diagonal matrix of units and $E$ is the remaining non-units with zeros on the diagonal. $D$ is clearly a unit. $E$ is in the Jacobson radical as it is a non-unit in a local ring. It follows that $1 + D^{-1}E$ is unit because "$x$ is in the Jacobson radical of $R$ iff $1-xr$ is a unit for all $r\in R$." Then $D(1+D^{-1}E) = D + E$ is the product of units, and thus a unit.