im reading a paper in which the author compute the determinant of this block matrix$$\left( \begin{array}{c|c|c|c|c} X & 0 & 0 & 0 & V_0 \\ \hline -I & X & 0 & 0 & V_1 \\ \hline 0 & -I & X & 0 & V_2 \\ \hline 0 & 0 & -I & X & V_3 \\ \hline 0 & 0 & 0 & -I & X+V_4 \end{array}\right) $$ and he get $\det(V_0+V_1x+V_2x^2+V_3x^3+V_4x^4+x^5)$using the schur compliment which i didn't understand .
so i tried to compute it otherwise and in the process i mistakenly use the determinant expansion by fifth column and get the correct result !!
so i thought the expansion must apply in case of block matrices and i look over the internet and did not find a proof,
sorry $I$ is an identety matrix and $X=x*I$
What you said isn't quite accurate. The determinant of that matrix is not $V_0+xV_1+x^2V_2+x^3V_3+x^4V_4+x^5I$ but $\color{red}{\det}(V_0+xV_1+x^2V_2+x^3V_3+x^4V_4+x^5I)$. In other words, to calculate the determinant of the large matrix, you first treat each sub-block as a scalar and calculate the "determinant" of a $5\times5$ matrix using "blockwise Laplace expansion". The resulting "determinant" is yet another matrix whose size is identical to the size of each sub-block. Take the determinant of this resulting matrix, you get the determinant of the original large matrix.
This blockwise Laplace expansion works because all sub-blocks in the first four block columns commute.
More specifically, consider a block matrix $$ M=\pmatrix{A_{11}&\cdots&A_{1n}\\ \vdots&&\vdots\\ A_{n1}&\cdots&A_{nn}}, $$ where all sub-blocks $A_{ij}$s are $m\times m$ matrices over some commutative ring $K$. If we remove the $i$-th block row and the $j$-th block column from $M$, we get an $m(n-1)\times m(n-1)$ submatrix over $K$, or an $(n-1)\times(n-1)$ matrix over $M_m(K)$ if you view each sub-block of $M$ as a matrix element. Let us call this matrix $\widehat{A}_{ij}$. E.g. $$ \text{if }\ M=\pmatrix{A_{11}&A_{12}&A_{13}\\ A_{21}&A_{22}&A_{23}\\ A_{31}&A_{32}&A_{33}}, \ \text{ then }\ \widehat{A}_{23}=\pmatrix{A_{11}&A_{12}\\ A_{31}&A_{32}}. $$ Now suppose we want to perform a blockwise Laplace expansion of this $M$ along the last column. Two issues immediately arise:
Fortunately, these issues go away if all sub-blocks outside the row/column that we want to expand along commute.
Note that when all sub-blocks of $\widehat{A}_{ij}$ commute, $Det(\widehat{A}_{ij})$ is well defined. Actually it is just the usual determinant, except that the underlying commutative ring is not $K$ but the ring generated by all sub-blocks of $\widehat{A}_{ij}$. E.g. in our previous example, when $A_{11}A_{32}=A_{32}A_{11}$ and $A_{31}A_{12}= A_{12}A_{31}$, it does not matter whether we calculate $Det(\widehat{A}_{23})$ as $A_{11}A_{32}-A_{12}A_{31}$ or $A_{32}A_{11}-A_{31}A_{12}$, because the two expressions are identical. So, the first issue is resolved.
Concerning the second issue, we define blockwise Laplace expansion as follows:
Then we have the following result for column expansion: $$ \det(M) = \det(L_{\ast j}(M))\ (\in K).\tag{$\dagger$} $$ The result for row expansion is similar. (In your particular case, $L_{\ast5}(M)=V_0+xV_1+x^2V_2+x^3V_3+x^4V_4+x^5I_m\in M_m(K)$ and $\det(M)=\det(L_{\ast5}(M))=\det(V_0+xV_1+x^2V_2+x^3V_3+x^4V_4+x^5I_m)\in K$. Since the sub-blocks in the first four columns also commute with those in the fifth one, it doesn't really matter if you put the pivot before or after the minor in $(1)$ here, but in general, you should put the minors before the pivots when you do a column expansion.)
This generalises the theorem (cf. John Silvester, Determinants of Block Matrices) that $\det\pmatrix{A&B\\ C&D}=\det(AD-CB)$ when $AC=CA$.
The proof of $(\dagger)$ is parallel to Silvester's proof. We first redefine $M$ as $M+yI$, where $y$ is an indeterminate adjoined to the ring $K$. By doing so, we can guarantee in our proof that the determinants or blockwise determinants of some proper principal submatrices are not zero divisors. Once the Laplace expansion formula is proved to be true for this new $M$, it is also true for the original $M$ by specialising $y$ to zero.
Now, suppose we want to expand along the last column and all sub-blocks off the last column commute. Denote by $R$ the commutative subring generated by the sub-blocks in the first $n-1$ block columns of $M$. Then $\sum_{i=1}^n(-1)^{i+n}Det(\widehat{A}_{in})A_{ij}=0$ for every $j<n$, because $\left((-1)^{1+n}Det(\widehat{A}_{1n}),\ldots,(-1)^{n+n}Det(\widehat{A}_{nn})\right)$ is just the last row of the adjugate matrix of $$ B=\begin{bmatrix}A_{11}&\cdots&A_{1,n-1}&0\\ \vdots&&\vdots&\vdots\\ A_{n-1,1}&\cdots&A_{n-1,n-1}&0\\ A_{n1}&\cdots&A_{n,n-1}&I_m \end{bmatrix}\in M_{n-1}(R) $$ and $Det(B)$ is not a zero divisor. It follows that $$ \left[ \begin{array}{ccc|c} I_m&&&0\\ &\ddots&&\vdots\\ &&I_m&0\\ \hline (-1)^{n+1}Det(\widehat{A}_{n1})&\cdots&-Det(\widehat{A}_{n,n-1})&Det(\widehat{A}_{nn}) \end{array}\right] \,M =\begin{bmatrix}\widehat{A}_{nn}&\ast\\ 0&L_{\ast n}(M). \end{bmatrix} $$ Take determinants on both sides, we get $$ \det(Det(\widehat{A}_{nn}))\det(M)=\det(\widehat{A}_{nn})\det(L_{\ast n}(M)).\tag{3} $$ However, by Silvester, if all sub-blocks of a block matrix $B$ commute, then $\det(Det(B))=\det(B)$. Therefore $(3)$ reduces to $\det(\widehat{A}_{nn})\det(M)=\det(\widehat{A}_{nn})\det(L_{\ast n}(M))$. Yet, $det(\widehat{A}_{nn})$ is not a zero divisor. Therefore we may cancel it out on both sides and get $\det(M)=\det(L_{\ast n}(M))$.