This is from exercise 3 of section 11.4:
Should those $x_i$ be in $R$ so that we have the vector $(x_1, \dots, x_n) \in V$?
This is from exercise 3 of section 11.4:
Should those $x_i$ be in $R$ so that we have the vector $(x_1, \dots, x_n) \in V$?
On
If we interpret the exercise as written, here's one solution . . .
Let $f(t)=\det(tI-A)$ be the characteristic polynomial of $A$.
In expanded form, we have $$f(t)=t^n + r_{n-1}t^{n-1}+ \cdots + r_1t+ r_0$$ for some $r_0,...,r_{n-1}\in R$.
Note that $r_0=f(0)=\det(-A)=(-1)^n\det(A)$.
Let $\vec{x}\in V^n$ be given by $$ \vec{x} = \pmatrix { x_1\\ \vdots\\ x_n\\ } $$ From the given equation $A\vec{x}=0$, it follows that $A^k\vec{x}=0$ for all positive integers $k$.
Applying the Cayley-Hamilton Theorem, we get \begin{align*} &f(A)=0\\[4pt] \implies\;&f(A)\vec{x}=0\\[4pt] \implies\;&(A^n + r_{n-1}A^{n-1}+ \cdots + r_1A+ r_0I)\vec{x}=0\\[4pt] \implies\;&r_0\vec{x}=0\\[4pt] \implies\;&(-1)^n\det(A)\vec{x}=0\\[4pt] \implies\;&\det(A)\vec{x}=0\\[4pt] \implies\;&\det(A)x_i=0\;\text{for all}\;i\\[4pt] \end{align*} as was to be shown.
But note: The Cayley-Hamilton Theorem is established in a later chapter.
If we restrict to results from chapters up to and including the chapter for the exercise in question, I don't see a way to solve the problem unless (as you suggested) we assume $V=R$.
With that assumption, we can argue as follows . . .
Suppose $A\vec{x}=0$, where $\vec{x}\in R^n$ is given by $$ \vec{x} = \pmatrix { x_1\\ \vdots\\ x_n\\ } $$ Let $A_1,...,A_n$ denote the columns of $A$, and let $B=x_1A_1 + \cdots + x_nA_n$.
From the equation $A\vec{x}=0$, it follows that $B=0$, hence by Cramer's Rule, we get $$ \det(A)x_i = \det(A_1,...,A_{i-1},B,A_{i+1},...,A_n) = \det(A_1,...,A_{i-1},0,A_{i+1},...,A_n) = 0 $$ for all $i$, as was to be shown.
I would say no, there is no mistake and the $x_i$ are supposed to be elements of $V$. I agree that it might be a bit strange to put module elements into a "vector", but the equation makes sense: they simply want say that you have $$\sum_{j = 1}^n A_{i,j} x_j = 0$$ for all $1 \leq i \leq n$. Also note that an arbitrary $R$-module is not necessarily of the form $R^n$.