Keras lstm equations, matrices and diagram

204 Views Asked by At

I was reading this post, mainly the LSTM equation section for understand the structure diagram and relate it with the matrices and I can deducted this cell diagram:

enter image description here

I made this homemade diagram because in the Colah's article it's not the exactly same cell that in keras.

I need know if it ok, can someone confirm me that? This are the equations

enter image description here

W are weights matrices for input data and U are weights matrices for previous hidden state

b are biases

In the pink circles there are operations. The point it's multiply element by element and X is matrix multiplication

1

There are 1 best solutions below

0
On

It's ok. The picture and all the formulas in the question describe a standard LSTM network. The part that confuses you may be the content of Keras code: such as dropout and recurrent dropout.

parameter settings dropout and recurrent_dropout

As long as you don't set these parameters when calling LSTM cells, you won't be confused about these advanced structures.