I have a system of $2$ linear equations, with $4$ variables $\{a,b,c,d\}$
$$e^{-\frac{n}{2}} \left(5 a e^n-b n+b\right)-5 c n+3 d m=0,$$ $$e^{-\frac{1}{2} (n-4)} \left(3 b e^n-5 a\right)+5 c e^{m n}-3 d e^{-m n}=0$$
$m,n$ are positive parameters.
I want to find the condition under which the system is solvable. I mean I want to obtain the condition when the determinant of the matrix of coefficients is zero! But we know that the matrix of coefficients must be a squared-matrix to be able to calculate the determinant, but in this case, it is not.
How can I obtain the condition of the solvability of the system in such cases? Or in general, is it possible to do that?
Presumably, by "solvable" you mean that the system has a non-zero solution. Your system is always solvable in this sense, because any system of linear equations in which the number of variables outnumbers the number of equations has a non-zero solution. In your case, you can write your equations in the form of a matrix equation: $$ \pmatrix{5e^{\frac{n}{2}}&-(n-1) e^{-\frac{n}{2}}&-5n&3m\\ -5e^{-\frac{(n-4)}{2}}&3e^{\frac{n+4}{2}}&5e^{mn}&-3e^{-mn}}\pmatrix{a\\b\\c\\d}=\pmatrix{0\\0}\ , $$ or, equivalently, $$ \pmatrix{5e^{\frac{n}{2}}&-(n-1) e^{-\frac{n}{2}}\\ -5e^{-\frac{(n-4)}{2}}&3e^{\frac{n+4}{2}}}\pmatrix{a\\b}= \pmatrix{5n&-3m\\-5e^{mn}&3e^{-mn}}\pmatrix{c\\d}\ . $$ The $\ 2\times2\ $ matrix on the left side of this last equation has determinant $$ 5\left(3e^{n+2}-(n-1)e^{2-n}\right)=5e^{2-n}\left(3e^{2n}+1-n\right) $$ which is strictly positive for all values of $\ n\ $. The matrix is therefore nonsingular, and you can solve the system by choosing $\ c\ $ and $\ d\ $ to have any values you please, and then solving the equations above to obtain $\ a\ $ and $\ b\ $ in terms of $\ c\ $ and $\ d\ $.