Kronecker product and outer product confusion

Question

I have two column vectors:

\begin{equation} u = \left[\matrix{ 1 \cr 2\cr }\right] \end{equation}

\begin{equation} v = \left[\matrix{ 4 \cr 4\cr }\right] \end{equation}

I'm trying to compute the Kronecker product of two vectors $u \otimes v$.

As I understand, the outer product of vectors is a special case of the Kronecker product of matrices.

http://en.wikipedia.org/wiki/Kronecker_product says:

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product $A \otimes\ B$ is the mp × nq block matrix.

http://en.wikipedia.org/wiki/Outer_product says:

So will $u \otimes v$ be of dimension 4 × 1 (according to the first definition) or 2 × 2 (according to the second definition)?

Answer 1

This is a very good example of abuse of notation, more precisely, reload of operator. Actually the operator $\otimes$ is usually used as tensor product, which is a bilinear operator. It's easy to verify that both Kronecker product (denoted by $\otimes_K$) and outer product (denoted by $\otimes_O$) are bilinear and special forms of tensor product. For example, given two vectors $u,v\in V$, we have $$u\otimes_O v=u\otimes_Kv^H$$ This is why wiki says outer product is a special case of Kronecter product.

Answer 2

The relation between outer product $\circ$ and kronecker product $\otimes$ . $${\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} = vec(({\textbf{a}}\circ {\textbf{b}})^T) = vec(({\textbf{a}}{\textbf{b}^T}_{I \times J})^T_{J \times I})_{JI \times 1}$$

Before taking vectorization you should do a transpose.

In case of 3 vectors, the resulted matrix as before is multiplied by each entry in c vector and each result is a frontal slice of a tensor. Notice that $.*$ is an element wise product between each element in $\textbf{c}$ vector and the resultant matrix of $(\textbf{a} \circ \textbf{b})$ and not the element wise product between the whole vector $\textbf{c}$ and $(\textbf{a} \circ \textbf{b})$. $$ {\textbf{a}}_{I \times 1} \otimes {\textbf{b}}_{J \times 1} \otimes {\textbf{c}}_{K \times 1} = vec(( ({\textbf{a}}\circ {\textbf{b}}) .* {\textbf{c}})^T_{I \times J \times K})_{IJK \times 1} $$

Outer product between 2, each of 2 dimension then the result would be 4 dimension tensor which is totally different than the kronecker product

Kronecker Product for matrices $$ {\textbf{A}}_{I \times R} \otimes {\textbf{B}}_{ J \times R} = {\textbf{D}}_{ IJ \times R^2} $$

$$ {\textbf{D}} = \begin{bmatrix} \vdots & \vdots & & \vdots & & \vdots \\ \textbf{a}_1 \otimes {\textbf{b}_1} & {\textbf{a}_1} \otimes {\textbf{b}_2} & \dots & \textbf{a}_1 \otimes {\textbf{b}_R} & \dots &\textbf{a}_R \otimes {\textbf{b}_R} \\ \vdots & \vdots & & \vdots & & \vdots \end{bmatrix}$$

Outer Product for matrices $$ \textbf{A}_{I \times R_1} \circ \textbf{B}_{J \times R_2} = {\textbf{C}}_{I \times J \times R_1 \times R_2}$$ to show how the result would be you have to do it column by column outer product, $$ {\textbf{a}_1} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{y}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{a}_1} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{n}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{p} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix},{\textbf{a}_2} \circ {\textbf{b}_2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{j} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix},$$

The final result would be, $${\textbf{C}}_{I \times J \times 1 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{x} & {\textbf{p}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 1} = \begin{bmatrix} \vdots & \vdots \\ \textbf{y} & {\textbf{q}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 1 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{m} & {\textbf{k}} \\ \vdots & \vdots \end{bmatrix}, {\textbf{C}}_{I \times J \times 2 \times 2} = \begin{bmatrix} \vdots & \vdots \\ \textbf{n} & {\textbf{j}} \\ \vdots & \vdots \end{bmatrix}$$