Let us consider an agent that can walk along a one-dimensional room (i.e. along a segment). The segment is split into three equal parts and we assume the agent is in the middle piece of the segment, as happens for example in the figure:
I would like to define a cost function for the agent as follows. The agent can move left or right, always moving only one position. If the agent is in any of the three parts of the segment, then he suffers no losses; if instead he ends up outside the segment he loses 1 dollar. For example, taking the previous figure as a reference, if the sequence of movements were "right, left, left, right, right" the agent would not suffer any losses.
On the other hand, if the agent's movement sequence is "right, right, left, left, right" then the loss is 1 dollar. We can also hypothesize that if the agent continues to cross the border, the losses accumulate; ie if for example the sequence of movements of the agent is "right right right" then the total loss of the agent is 2 dollars (and so on).
To find the mathematical expression of this cost function, I thought of indicating the agent's action (that is, whether to go left or right) with the letter $x$. So: $x\in\{L,R\}$. Moreover, $x$ depends on time step $t$. But now how could I represent the penalty that occurs after having trespassed outside the segment?
For a given sequence $x\in\{R,L\}^n$ let $P_x(i)=|\{j\le i:x_j=R\}|-|\{j\le i:x_j=L\}|$ be the position at step $i$. Then the loss function with no accumulation is the number $|\{i:|P_x(i-1)|=1,|P_x(i)|=2\}|$ of times we leave the segments (assuming the costs do accumulate when we leave several times), and with accumulation it is the number $|\{i:|P_x(i)|>1\}|$ of times that we are outside.