We define an instantaneous description of a Turing machine as a word $\alpha p \beta$, with $\alpha, \beta \in \Sigma^{*}$ and $q \in Q$, s.t. if
$$ d = \alpha_1 \ldots \alpha_n q \beta_1 \ldots \beta_n $$
we read "the Turing machine is at state $q$, with $\alpha, \beta$ on the tape and its head over $\beta_1$". Thus, an instantenous description describes what is in the tape of the Turing machine, where the machine's head is, and what state the machine is in.
We say $d \vdash d'$ if and only if
- $d$ contains $q$ and $d'$ contains $q'$, each a state of $Q$;
- if the machine's description is $d$, then after $\delta(q)$ the machine's description is $d'$.
I'll provide an example to make sure this is clear:
Let $\Sigma = \{ @, \#\}$. Assuming $\delta (p) = \{ (q,\#, K) \}$.
\begin{align*} &@ ~ \# ~ @ ~ p ~ @ ~ @ ~ @ ~ B ~ B ~ B \ldots \vdash @ ~ \# ~@ ~ q ~ \# ~ @ ~ @~ B ~ B ~ \ldots \end{align*} where $B$ is the blank symbol.
One very important point is that, for a given instantaneous description $\alpha q \beta$, $\vdash$ is not defined if $\delta(q) = \{(q', \sigma, L\}$ and $\alpha = \varepsilon$. Informally, the machine cannot move left if there is no symbol to the left of the current head's position.
I was asked to prove the following statement:
Prove that $M$ is deterministic iff for each instantaneous description $d$ there is at most one instantaneous description $d'$ s.t. $d \vdash d'$.
This statement seems to be false! Consider the $(\Leftarrow)$ case; this is, assume that for each description $d = \alpha q \beta$ there is at most one $d'$ s.t. $d \vdash d'$. Consider the case where there is no $d'$ s.t. $d \vdash d'$. In the case where $\alpha = \varepsilon$ and
$$ \delta(q) = \{ (q', \sigma, L), (q'', \sigma', L) \} $$
the machine is non-deterministic without violating the assumption. In other words, that $d \vdash d'$ for at most one $d'$ does not seem to imply that the machine is deterministic.
You are quite right. You can add arbitrary unreachable states to the description of a Turing machine without affecting whether or not it is deterministic. Also the notion of a reachable state is undecidable, so there can be no effective test for whether a Turing machine is deterministic. It is true that, if there at most one successor state for any instantaneous description, then the Turing machine is deterministic.