i would like to prove that the matrix representation of the map $\phi:\mathbb{H}\to \mathbb{H}$, where $\phi(v)=pvp^{*}$ is an element of $SO(3)$,with respect to the basis $(\textbf{i},\textbf{j},\textbf{k})$ for the set of imaginary quatérnions, here $p$ is a unit quaternion.
I calculated $\phi$ in the elements of the base ($\textbf{i}$,$\textbf{j}$,$\textbf{k}$), here $p=(a,b)$ is a unit quaternion. Then
$\phi(\textbf{i})=(|a|^{2}-|b|^{2})\textbf{i}-2ab\textbf{k}$
$\phi(\textbf{j})=(-\bar{b}a-b\bar{a})\textbf{1}+(a^{2}+b^{2})\textbf{k}$
$\phi(\textbf{k})=(b\bar{a}+\bar{b}a)\textbf{i}+(a^{2}-b^{2})\textbf{k}$
the result of $\phi(\textbf{j})$ confuses me because there is a term that does not belong to the base $(-\bar{b}a-b\bar{a})\text{1}$
Any tips?
This answer is just an expansion of what anomaly pointed out, since you seem confused by his answer in the comments.
This is in reality a problem about linear algebra only:
Your map $\phi$ is clearly a isometry in $\mathbb{H}$, since \begin{align*} \langle pvp^*, pv'p^*\rangle &=\operatorname{Re}(pvp^*pv'^*p^*) \\ &=\operatorname{Re}(pvv'^*p^*) \\ &\stackrel{!}{=}\operatorname{Re}(v'^*p^*pv) \\ &=\operatorname{Re}(v'^*v) \\ &=\langle v,v' \rangle. \end{align*} Important: The equation $(!)$ is not using commutativity of the elements. It is using the fact that $\operatorname{Re}(xy)=\operatorname{Re}(yx)$.
Since $\phi(1)=1$, it thus follows that $\phi$ restricts to the orthogonal complement of $1$, which is a three-dimensional inner product space. Given any inner-product equipped vector space of dimension $n$, "its" $SO_V$ (let's call it that for the sake of clarity), i.e., the set of linear maps such that $\langle Av,Aw \rangle=\langle v,w \rangle$ (let's call this equation $(1)$) is always isomorphic to $SO(n)$, which you seem to define as a matrix group, via the matricial representation with respect to a fixed orthonormal basis $\mathcal{B}$ in $V$. This is simply a consequence of the fact that the matrix group $SO(n)$ can be characterized as the group of those matrices with orthonormal columns in the standard inner-product of $\mathbb{R}^n$. The map \begin{align*} SO_V &\to SO(n) \\ A &\mapsto A_{\mathcal{B}}, \end{align*} which sends a linear map to its matricial representation w.r.t $\mathcal{B}$ is obviously an isomorphism (the fact that it sends indeed to $SO(n)$ is immediate from equation $(1)$ and the characterization of the collumns of the matricial representation of a linear map)*. This is what you are doing when you say "$\phi$ (...) with respect to the basis $(i,j,k)$": just taking $\mathcal{B}:=\{i,j,k\}$.
So, to sum up: from the fact that $\phi(1)=1$ and $\phi$ is an isometry, we get that $\phi|_{1^{\perp}}:1^{\perp} \to 1^{\perp}$ is a well-defined map, which is by definition an element of $SO_{1^{\perp}}$. The base $\{i,j,k\}$ is obviously orthonormal, and thus the map above indeed sends us to $SO(3)$.
*- Just for the sake of completeness, letting $\mathcal{B}=\{f_1,\cdots,f_n\}$, the computation is : \begin{align*} \sum_{k=1}^na_{ki}a_{kj}&=\langle \sum_{l} a_{li}f_l, \sum_{m} a_{mj}f_m \rangle \\ &=\langle A f_i, Af_j \rangle \\ &=\langle f_i, f_j \rangle \\ &=\delta_{ij}. \end{align*}