I have a recursive vector-valued function
$$\mathbf{y}(t)=\mathbf{W}\mathbf{y}(t-1).$$
To compute the derivative of $\mathbf{y}(t)$ with respect to $\mathbf{W}$, do I need to use the product rule? i.e., is the derivative
$$\frac{d\mathbf{y}(t)}{d\mathbf{W}} = \mathbf{W}\frac{d\mathbf{y}(t-1)}{d\mathbf{W}},$$
or
$$\frac{d\mathbf{y}(t)}{d\mathbf{W}} = \mathbf{W}\frac{d\mathbf{y}(t-1)}{d\mathbf{W}}+\mathbf{y}(t-1)?$$
Ignore the $t$-dependence and consider this linear equation and its differential $$\eqalign{ y &= W \cdot x \cr dy &= dW \cdot x \cr &= I\cdot dW \cdot x \cr &= (Ix):dW \cr }$$ where $(Ix)$ is a third-order tensor with components $(Ix)_{ijk}=\delta_{ij}x_k$. I've written an explicit dot for the dot product, so as to reserve juxtaposition to denote the tensor product. Also note that the colon denotes the double-dot product, $A\!:\!B = \sum_{jk}A_{ijk}B_{jk}$
Anyway, since $dy=(\frac{\partial y}{\partial W}):dW$, the derivative is $$\eqalign{ \frac{\partial y}{\partial W} &= Ix \cr }$$