A matrix can be regarded as a representation of a linear function from one vector space to another, with the condition that the basis vectors (and their order) are specified for both spaces.
Therefore a column matrix must necessarily be a representation of a vector of a space in relation to that space basis.
For example, let $\vec{u}=4\vec{e_1} + 1\vec{e_2}$. In matrix notation we have $\left[ \begin {matrix} 4 \\ 1 \\ \end {matrix} \right] = 4 \left[ \begin {matrix} 1 \\ 0 \\ \end {matrix} \right] + 1 \left[ \begin {matrix} 0 \\ 1 \\ \end {matrix} \right] $.
Let's assume the basis ($\vec{e_1}, \vec{e_2}$) looks like the vectors shown in the figure (The basis vectors and their linear combination).
My question is: what is the dot product of the basis vectors?
According to the matrix notation it should be zero: $ \left[ \begin {matrix} 1 \\ 0 \\ \end {matrix} \right] \cdot \left[ \begin {matrix} 0 \\ 1 \\ \end {matrix} \right] =0 $. But, according to the figure the dot product should not be zero as both vectors are clearly not orthogonal.
Obviously I am missing something. Any help would be appreciated.
While the dot product in $\Bbb R^n$ is defined by the coordinates of the vectors with respect to the canonical basis, it is to be considered a function of the vectors, not of their coordinates. Thus when you change to a different basis, you do not change the value of their dot product. Instead, the dot product is expressed by a different formula. A concrete example may help.
In $\Bbb R^2$, let $\vec d_1 = (1,0), \vec d_2 = (0,1)$ be the canonical basis elements. And define $\vec e_1 = (1,1), \vec e_2 = (0,1)$, then $\vec e_1, \vec e_2$ are a non-orthogonal basis. Indeed $\vec e_1$ is not even normal: its length is $\sqrt 2$. Now let $\vec u = (u_1, u_2), \vec v = (v_1, v_2)$. We can express $\vec u, \vec v$ in terms of the $\vec e_1, \vec e_2$ basis: $$\vec u = u_1'\vec e_1 + u_2'\vec e_2\\\vec v = v_1'\vec e_1 + v_2'\vec e_2$$ If we solve the equations, it turns out that $u_1' = u_1$ and $u_2' = u_2 - u_1$, and similarly for $\vec v$. Now,
$$\begin{align}\vec u \cdot \vec v &= (u_1'\vec e_1 + u_2'\vec e_2) \cdot (v_1'\vec e_1 + v_2'\vec e_2)\\&=u_1'v_1'(\vec e_1 \cdot \vec e_1) + u_2'v_1'(\vec e_2 \cdot \vec e_1) + u_1'v_2'(\vec e_1 \cdot \vec e_2) + u_2'v_2'(\vec e_2 \cdot \vec e_2)\\&=u_1'v_1'(2) + u_2'v_1'(1) + u_1'v_2'(1) + u_2'v_2'(1)\end{align}$$
So in $\{\vec e_1, \vec e_2\}$ coordinates $$\begin{bmatrix}u_1' \\ u_2'\end{bmatrix}\cdot \begin{bmatrix}v_1' \\ v_2'\end{bmatrix} = \begin{bmatrix}u_1' & u_2'\end{bmatrix}\begin{bmatrix}2&1\\1&1\end{bmatrix}\begin{bmatrix}v_1' \\ v_2'\end{bmatrix}$$ while in the canonical $\{\vec d_1, \vec d_2\}$ coordinates $$\begin{bmatrix}u_1 \\ u_2\end{bmatrix}\cdot \begin{bmatrix}v_1 \\ v_2\end{bmatrix} = \begin{bmatrix}u_1 & u_2\end{bmatrix}\begin{bmatrix}1&0\\0&1\end{bmatrix}\begin{bmatrix}v_1 \\ v_2\end{bmatrix}$$