I have a similar problem as in this question.
In short words: Assume a square, positive semidefinite matrix $A\in\mathbb R^{n\times n}$. Show that if a diagonal element of $A$ is zeros then the corresponding row and column are all zero.
The answer in the other question suggests to look at the definiteness of the matrix $$ B = \begin{pmatrix} a_{i,i} & a_{i,k} \\ a_{k,i} & a_{k,k} \end{pmatrix}$$ where $a_{k,k}=0$ and both $a_{i,k}$ and $a_{k,i}$ are non-zero.
I tried to proof this fact but I am unsure if I did it correctly. Also there might be a better/quicker/clearer way to do it. Would you mind going through it and give me your oppinion?
If one calculates the eigenvalues we get $$\lambda_{1,2} = \frac{a_{i,i}\pm\sqrt{a_{i,i}^2+4a_{i,k}^2}}{2}.$$ Thus we know that $B$ is indefinite as $\sqrt{a_{i,i}^2 + 4a_{i,k}^2} > \lvert a_{i,i} \rvert$.
If $A$ is positive semidefinite then $x^T A x \geq 0$ for all $x\in\mathbb R$ must hold. Assume a matrix $C\in\mathbb R^{n \times 2}$ that is zeros except for the elemets $c_{i,1}=c_{k,2}=1$. This must also hold true for the special case that $x=Cy$ with $y\in\mathbb R^2$. We obtain $y^T C^T A C y = y^T B y$ means that $C^T A C$ is indefinite. This is a contradiction to the assumption that $A$ is positive semidefinite.
Is there an (easy) way to extend this to block matrices?
Your last sentence is a bit confusing, but I think I understand your overall argument. You show that a non-zero entry on the row/column implies the existence of an indefinite sub-matrix, which in turn implies the larger matrix is indefinite. If that's what you meant, then I would say your argument is in principle correct, although it could be written up in a clearer way. I am also not too sure what you had in mind when you say "extend to block matrices".
Here would be a cleaner write up of what I think you had in mind.
Let $A$ be an $n\times n$ symmetric positive-semidefinite matrix with a zero diagonal entry. By permuting the rows and columns of the matrix, we may assume without loss of generality that entry $a_{11}=0$. Suppose there is an entry in the first row/column which is non-zero, say $a_{1k}$. Then the submatrix $$A_{1k} = \begin{pmatrix}0 & a_{1k} \\ a_{1k} & a_{kk} \end{pmatrix}$$ is not positive-semidefinite (determinant is negative). Therefore there exists some non-zero vector $\mathbf{x'} = (x_1,\ x_2) \in \mathbb{R}^2$ such that $$\mathbf{x'}^\mathrm{T}A_{1k}\mathbf{x'} < 0$$ Now consider the vector $\mathbf{x}\in \mathbb{R}^n$ which has first entry $x_1$, $k$th entry $x_2$ and is zero everywhere else. Then $$\mathbf{x}^\mathrm{T}A\mathbf{x} = \mathbf{x'}^\mathrm{T}A_{1k}\mathbf{x'} < 0$$ contrary to the assumption that $A$ is positive-semidefinite.