A matrix $(a_{ij})_{i,j=1,\dots,n} $ with real nonnegative entries is called (row-)stochastic iff the row sum is equal to one $\sum_{j=1}^n a_{ij}=1$.
We'll call a matrix $T$ an infinitesimally stochastic matrix iff the matrix exponential $exp(tT)$ is a stochastic matrix for all $t\geq 0$.
In general, are there characterizations on these kind of matrices?
In particular, for which numbers $t$ do we have that $$T(t)=\begin{pmatrix} -t&t&0&0&\cdots&0\\ 0&-t&t&0&\cdots&0\\ \vdots&\ddots &\ddots&\ddots&\vdots&\vdots\\ 0&\cdots&\cdots&0&-t&t\\ 0&\cdots&\cdots&\cdots&\cdots&0\\ \end{pmatrix}$$
is an infinitesimally stochastic matrix. ($T=-tD + tJ$ where $D$ is a diagonal matrix with $1$s in the diagonal except in the last entry where it is zero and $J$ is a standard Jordan Block).
Numerically, this seems to be the case for a lot of $t$s, but I'm interested of the reason why this is the case. Maybe you can interpret $T$ as some kind of characteristic function for the multivariate multicnomial distribution that is represented by a stochastic matrix, but I'm also open for literature suggestions/other names for this kind of matrix.
A stochastic matrix is a matrix $M$ with real nonnegative entries satisfying $M v = v$ where $v$ is the all-ones vector. If $A$ is infinitesimally stochastic then in particular
$$\exp (At) v = v$$
and differentiating with respect to $t$ gives $Av = 0$, so a necessary condition is that the row sums are all zero. Next we need a condition on $A$ guaranteeing that $\exp (At)$ has real nonnegative entries. It's necessary and sufficient for the off-diagonal entries of $A$ to be nonnegative; to see that this is sufficient use the limit
$$\exp (At) = \lim_{n \to \infty} \left( 1 + \frac{At}{n} \right)^n$$
and to see that it's necessary take $t$ to be small enough that $\exp (At) \approx 1 + At$.
Matrices with these properties are called transition rate matrices, infinitesimal generators (of continuous-time Markov chains), or Q-matrices. In particular, the matrix you wrote is such a matrix.