Assuming that A is the matrix of a linear operator F in S find the matrix B of F in R
A is: \begin{bmatrix}1&1\\1&2\end{bmatrix}
S = {(1,1), (1,2)}
R = {(1,0), (0,1)}
F(1,1) = (1,1) + (1,2)
F(1,2) = (1,1) + 2(1,2)
F(2,3) = 2(1,1) + 3(0,1)
F(3,5) = 3(1,0) + 5(0,1)
So B=
\begin{bmatrix}2&3\\3&5\end{bmatrix}
Is that correct?
No. In order to compute $B$, you must compute $F(1,0)$ and $F(0,1)$. Since$$(1,0)=2(1,1)-(1,2),$$you know that\begin{align}F(1,0)&=2F(1,1)-F(1,2)\\&=2\bigl((1,1)+(1,2)\bigr)-\bigl((1,1)+2(1,2)\bigr)\\&=(1,1).\end{align}So, $B$ will be like $\left(\begin{smallmatrix}1&?\\1&?\end{smallmatrix}\right)$. Now, you can compute the second column by the same process.