I have to determine the order of convergence of this iteration specification but I don't know how to do this in $\mathbb{R}^n$. (So far, I've only done this in $\mathbb{R}$). Any help is highly appreciated (and sorry for my bad english).
$x^{(k+1)} = \begin{pmatrix} 0 & -1/2 & -1/4\\ -1/2 & 0 & 0\\ 0 & -1/2 & 0 \end {pmatrix}x^{(k)}+\begin{pmatrix} 3/4 \\ -2 \\ -1/2 \end{pmatrix}$ for $k \geq 0$
$x^{(k)} = \begin{pmatrix} x_1^{(k)} \\ x_2^{(k)} \\ x_3^{(k)} \end{pmatrix}$
Your iteration is special case of the stationary iteration $$x_{n+1} = Gx_n + f$$ which can occasionally be used to solve the linear system $$x = Gx +f.$$ The initial guess $x_0$ must be selected by the user, but $x_0 = 0$ is a perfectly acceptable choice. If $\|G\|_2<1$, then $I-G$ is nonsingular, and the sequence $\{x_n\}_{n=0}^\infty$ is convergent and $$x_n \rightarrow x = (I-G)^{-1} f, \quad n \rightarrow \infty, \quad n \in \mathbb{N}$$ regardless of the choice of $x_0$. If $x_0 = 0$, then by induction on $n$ you can establish that $$x_n = \sum_{j=0}^{n-1} G^j f.$$ It follows, that $$\|x-x_n\|_2 \leq \left \| \sum_{j=n}^\infty G^j f \right\|_2 \leq \|G\|_2^n\|x\|_2 = \epsilon_n.$$ It follows that $x_n \rightarrow x$ at least linearly as $n \rightarrow \infty$ and $n \in \mathbb{N}$ because $$ \frac{\epsilon_{n+1}}{\epsilon_n} = \frac{ \|G\|_2^{n+1} }{\|G\|_2^{n}} = \|G\|_2 \rightarrow \|G\|_2 < 1.$$ You are free to replace the $2$-norm with any other norm induced by a vector norm. In particular, you can use the $\infty$-norm for which your particular matrix $G$ satisfies $\|G\|_\infty = \frac{3}{4}$.
In any case, the order of convergence is at least $p=1$.