What is the correct order of multiplication for this product of matricies?

Question

I am groking the following equation:

$$ \frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}} = \prod_{i = k + 1}^{t}\mathbf{\Theta}^T \text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})] $$

Where $\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}} $ is a 2 x 2 jacobian matrix, $\mathbf{\Theta}^T$ is a 2 x 2 matrix, and $\text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})]$ is also a 2 x 2 matrix.

My question is, in what order should this multiplication be executed? I know that matrix multiplication is not commutative in general, and so I am stumped as to what "order" I am supposed to perform the above matrix products in. Does the equation dictate the order? If not, then how does one know?

Answer 1

You should consider this to be taken in the order that the index of the product says. So first $i=k+1$, then $i=k+2$, then $i=k+3$ and so on until $i=t$. This expansion occurs left to right, with $i=k+1$ on the far left and $i=t$ on the far right.

Answer 2

Matrix multiplaction is not commuative, but it is associative. This means that $A \cdot B \cdot C$ is different from $B\cdot A \cdot C$. However, $(A \cdot B) \cdot C$ is the same as $A \cdot (B \cdot C)$.

Answer 3

Matrix multiplication is associative, so you could take this in "pairs" then go in the proper ordering, right to left. However, since you have some diagonal matrices, multiplying a chain of those is special in that the order doesn't matter.

Here's an extended explanation. We know the following for matrices $A, B,$ and $C$ of suitable sizes (the multiplication makes sense):

$$ (AB)C = A(BC) $$

yet in general, $AB \neq BA$. So, when you have a chain, you can work in pairs - here's an example:

$$ (AB)(CD) = A(BC)D $$

(again, assuming everything is of suitable size). Now let's examine matrix-vector multiplication in a chain.

\begin{align*} ABCD\mathbf{x} &= ABC\mathbf{x'} \\ &= AB\mathbf{x''}\\ &= A\mathbf{x'''} \end{align*}

This is if we use the usual "right-to-left" scheme. Of course, we may also multiply any two adjacent matrices and still arrive at the same product.

The commutativity of diagonal matrix multiplication is very easy to see - take any two diagonal matrices of the same size and multiply both ways. It's neat to see!