Let $\textbf{P}_{2}(\textbf{R})$ be the real vector space of polynomials of degree less than or equal to $2$.
Let $f:\textbf{P}_{2}(\textbf{R})\rightarrow \textbf{P}_{2}(\textbf{R})$ be the linear map given by differentiation, i.e., $f(p(x)) = p'(x)$.
Compute the matrix of $f$ with respect to the basis $\{1, x + 1, x^2 + x\}$.
MY ATTEMPT
So far I've done $f(1) = 0$, $f(x+1) = 1$ and $f(x^2 + x) = 2x + 1$.
On
Since we are dealing with a linear operator, we can assume the same basis for the domain and the counterdomain.
Given the basis $\mathcal{B} = \{1,x+1,x^{2}+x\}$, we have the following system of equations to solve \begin{align*} \begin{cases} a_{11} + a_{12}(x+1) + a_{13}(x^{2}+x) = f(1) = 0\\\\ a_{21} + a_{22}(x+1) + a_{23}(x^{2} + x) = f(x+1) = 1\\\\ a_{31} + a_{32}(x+1) + a_{33}(x^{2} + x) = f(x^{2}+x) = 2x + 1 \end{cases} \end{align*} whence we conclude $f(1) = (0,0,0)_{\mathcal{B}}$, $f(x+1) = (1,0,0)_{\mathcal{B}}$ and $f(x^{2}+x) = (-1,2,0)_{\mathcal{B}}$.
Finally, one has that \begin{align*} [f]_{\mathcal{B}} = \begin{bmatrix} 0 & 1 & -1\\ 0 & 0 & 2\\ 0 & 0 & 0 \end{bmatrix} \end{align*}
and we are done.
Hopefully this helps.
Let's consider the first equation: $$ f(1) = 0 $$ Now, we can decompose the input and the output on the polynomial basis you suggested: $$ f(1\times(1) + 0(x+1) + 0(x^2+1))= 0\times(1) + 0(x+1) + 0(x^2+1) $$ Reading out the coefficients both on the input and the output, we deduce that the sought matrix, call it $f_{mat}$, must be such that $$ f_{mat}(1,0,0)^T = (0,0,0)^T $$ This equation is equivalent to state that the first column of $f_{mat}$ is $(0,0,0)^T$. We can proceed similarly for the other two equations: $$ f_{mat}(0,1,0)^T = (1,0,0)^T \\ \text{i.e. } f(0\times(1) + 1(x+1) + 0(x^2+1))=1\times(1) + 0(x+1) + 0(x^2+1) \\ f_{mat}(0,0,1)^T = (-1,2,0)^T \\ \text{i.e. } f(0\times(1) + 0(x+1) + 1(x^2+1))=-1\times(1) + 2(x+1) + 0(x^2+1) $$ Thus obtaining $$ f_{mat}= \begin{pmatrix} 0 & 1 & -1 \\ 0 & 0 & 2 \\ 0 & 0 & 0 \\ \end{pmatrix} $$