$$L = \{\langle M \rangle |\ M\ \text{is a}\ TM\ \text{and}\ \exists x\ \text{s.t.}\ |x|<5\ \text{and}\ M\ \text{rejects}\ x\}$$ Can I use Rice Theorem in order to prove $L \notin R$?
I'm confused since $L$ contains a details about the machine behvior (rejects under some case). but I can find a language $C = \{ L | L \in RE ,\exists x \notin L, |x| < 5 \} $ such that it is non-trivial and equivalent to the machine language.
Am I using Rice theorem correctly?
In this case the details about the machine behaviour only mean that there exists a string that does not belong to the language recognized by the machine. In other words, $L$ can be rewritten as $$L = \{\langle M \rangle |\ M\ \text{is a}\ TM\ \text{and}\ \exists x: |x|<5\ \text{and}\ x \notin L(M)\}$$ Now it is easy to see that $L$ satisfies the two conditions of Rice's theorem.
Consequently, Rice’s theorem implies that $L$ is undecidable.