This is the schematic for an LSTM neuron, given in Wikipedia
I have question on output gate. It passes the cell state trough tanh function, which maps the value to an open interval between -1 and 1. Then it passes the current observation (on which we train the network) and the previous hidden state to logistic function that maps these values to open interval between 0 and 1. Then we multiply the output of both functions to get ... something in the open interval between -1 and 1.
If this is correct, the LSTM neuron cannot predict values outside the interval between -1 and 1. And the literature speaks something about Hadamard product, which implies that some of the states can be vectors or matrices. But even then it seems that their elements cannot be outside the interval.
If this is correct, we cannot predict upward trend, i.e. one that surpasses our maximal observation in the training data. Even if we map the input to interval between 0 and 1, this would require the neuron to output something above 1.
So my question is: It's obvious that there is a problem with my "mental model" of LSTM. Can you point out where is my mistake?