Assume that we have a random vector $X$ taking values from $\mathbb{R}^{k}$ which follows a multivariate normal distribution, i.e. $X \sim \mathcal{N}(\mu, \Sigma)$, where $\mu \in \mathbb{R}^{k}$ is some constant and $\Sigma$ is $k \times k$ positive semi-definite matrix.
Next, let us consider a random vector $X' \in \mathbb{R}^{k+1}$ contracted from $X$ with one more constant element, say $c$, added to its end. From the definition of multivariate random vector, the vector $X'$ is also a normal random vector.
What is its covariance matrix in this case? Would it be correct to say (and write) that $\Sigma'$ is $(k+1)\times (k+1)$ matrix which is equal to $\Sigma$ with added the last raw and column of all zeros, i.e.
\begin{equation*} \Sigma' = \begin{pmatrix} \Sigma & \mathbf{o} \\ \mathbf{o}^{T} & 0 \end{pmatrix}, \end{equation*} where $\mathbf{o} \in \mathbb{R}^{k}$ is a vector of zeros.
PS I have never seen a covariance matrix with a diagonal element equal to zero.
The last element in the diagonal of $\Sigma'$ would normally be equal to $0$ since the last element of the variable $X_n'$ is constant.
This comes from the covariance formula, where we have $$var(c,c) = E[(c-E(c))(c-E(c))]$$
Since $c$ is constant we also have: $$E(c) = c$$
So $$var(c,c)=0$$
I hope this answers your question.