What kind of examples of groups and representations should I keep in mind, which do not uniquely decompose into irreducible representations? I am mostly interested in characteristic zero representation, but I am also curiuous about positive characteristic.
What about the traceability of such representation? What algebraic structure give these issues? For which groups, we do not have such issues?
You have to be careful about the difference between irreducibles and indecomposibles. The represenation $\mathbb{Z}\to GL_2$ defined by $1 \mapsto \left( \begin{array}{cc} 1& 1\\ 0&1\end{array}\right)$ is the basic example of a represenation that is reducible but doesn't decompose into irreducibles.
On the other hand if you're asking when things split up into indecomposibles, there is the Krull-Schmidt theorem which says that if a module satisfies both the ascending chain condition and the descending chain condition, then it is a direct sum of indecomposibles. So in particular this would include any finite dimensional representation $G\to GL_n(\mathbb C)$
For counterexamples, maybe you could look at this book, I just now found it by google searching 'krull-schmidt counterexample'... the first example they give is that for $R= \mathbb{Z}[\sqrt{-5}]$ you have $\langle 3,2+\sqrt{-5}\rangle \oplus \langle 3,2-\sqrt{-5}\rangle \cong R\oplus \langle 3\rangle$, so the decompositions aren't unique in this case in the sense that the summands on the LHS and RHS aren't isomorphic as $R$-modules. But this is a statement about rings rather than group representations, so it might not be quite what you're after...