Consider a second order PDE, \begin{eqnarray*} au_{tt}+bu_{tx}+cu_{xx} & = & 0, \end{eqnarray*} which is equivalent to the following first order PDEs (introducing new variables $p$ and $q$) \begin{eqnarray*} u_{t}-p & = & 0,\\ u_{x}-q & = & 0,\\ ap_{t}+bp_{x}+cq_{x} & = & 0. \end{eqnarray*} I feel confused about the characteristic determinant of them. The principal part of the above first order PDEs is given by \begin{eqnarray*} \left(\begin{array}{ccc} \partial_{t} & 0 & 0\\ \partial_{x} & 0 & 0\\ 0 & a\partial_{t}+b\partial_{x} & c\partial_{x} \end{array}\right)\left(\begin{array}{c} u\\ p\\ q \end{array}\right) \end{eqnarray*}
Then the characteristic determinant of the first order PDEs is always zero. However, the characteristic determinant of the original second order PDE can be nonzero for suitable $a,b$, and $c$. So, how to understand this? Can you give me some friendly references? Thanks!
The issue here is that you have not written down all of the first order equations satisfied by $u, p, q$. What is missing is $$q_t - p_x = 0.$$ This is now an overdetermined system, since there are 4 equations for 3 unknown functions. It is indeed possible, as you have done, to choose a 3-by-3 subsystem that has a degenerate symbol. However, given suitable assumptions on $a, b, c$, it is possible to choose one that is not.
If your intention is to solve an initial value problem for the second order equation, where $u, u_t$ at $t=0$ is given, then it is necessary to assume $a \ne 0$. The appropriate first order subsystem to use is \begin{align*} u_t - p &= 0 \\ q_t - p_x &= 0\\ ap_t + bp_x + cq_x &= 0 \end{align*} This system has a nondegenerate symbol and, for example, if the coefficients and the initial data are real analytic, the initial value problem can be solved using the Cauchy-Kovalevski theorem. If the system is hyperbolic, then the initial value problem can be solved with smooth initial data.
To solve the second order equation with $u$ and $u_t$ specified at $t = 0$ using the first order system, you would use the following initial data: $u$ as given, $p = u_t$ as given, and $q = u_x$. You do, however, have to verify that the solution satisfies $u_x = q$ holds for $t \ne 0$, since it is not directly implied by the subsystem.