Using derivative to estimate change overestimates change between two states

Question

I have a profit function in which revenue is given by the state of a variable y at each point in time t: R=y[t]-y[t]^2 and costs depend on the change in y from the previous state so that large changes are more costly C=(y[t]-y[t-1])^2.

[Edit for clarification: the model was originally given to me in discrete time as above. I changed it to continuous time by replacing the difference in the cost term with the derivative of y. After optimizing this, I got the optimal path of y as a continuous function] In this case, t goes from 0 to 2 and y is between 0.2 and 0.5. If I look at the values of the y points and the changes between them, the derivative is much larger than the actual change, depending on how small I set the time intervals (e.g. 0.1 or 0.01).

Because the derivative is much larger than the change, the costs explode and I get negative profits whereas if I calculate the change between the y the profits are always positive because changes are quite small.

Here is an example of what I am talking about, with p1 being calculated using the derivative and p2 using the change between y values:

   t    dy/dt   y(t)       p1     p2
0,00    0,60    0,20        
0,01    0,59    0,21    -0,20   0,16
0,02    0,57    0,21    -0,18   0,17
0,03    0,56    0,22    -0,16   0,17
0,04    0,54    0,22    -0,14   0,17
0,05    0,53    0,23    -0,12   0,18
0,06    0,52    0,23    -0,10   0,18
0,07    0,50    0,24    -0,09   0,18
0,08    0,49    0,24    -0,07   0,18
0,09    0,48    0,25    -0,06   0,19
0,10    0,47    0,25    -0,04   0,19

Am I totally off base using the derivative for this purpose? How can I include the rate of change in the cost function in a continuous model? Is the fact that I am going back and calculating profits using the discrete function the problem or is the use of the derivative?

If it is of any help, I am using optimal control theory to try to maximize the profit function using the rate of change as the control variable. The functions that I am using are the results of that procedure.

Thanks!

Answer 1

The way your problem is set up, the time dimension is discrete, and $y(t)$ apparently is continuous at each $t$, NOT from $t-1$ to $t$.

$$\pi_t = R_t - C_t = y_t-y_t^2 - (y_t-y_{t-1})^2 $$

For a continuous-time formulation it should be written

$$\pi(t) = R(t) - C(t) = y(t)-y(t)^2 - \dot y(t)^2 $$

Now if your problem is set up in continuous time, and you want to make specific numerical calculations, of course you will work with discrete intervals, where the value of the derivative should be approximated. In that case, remember that $\frac {dy}{dt}$ is "the change in $y$ IF $t$ changes by 1" (so the denominator $dt=1$) - not by "one (arbitrary) unit". In your case, $t$ does not change by $1$ but by $0,01$.

Answer 2

Try the following continuous time representation where $R$ and $C$ represent cumulative revenues and costs:

$R(t)=R(0)+\int_0^t{\{y(\tau)-y(\tau)^2\}\mathrm{d}\tau}$

$C(t)=C(0)+\int_0^t{\{\mathrm{d}y(\tau)/\mathrm{d}\tau\}^2\mathrm{d}\tau}$

Since revenue and costs depend only on instantaneous $y$ and $\dot{y}$, this representation is economically more applealing but won't exactly reproduce the discrete time numbers, which assign special importance to integer values of $t$.

For solution in discrete time, here is an approach. A solution $y_n\equiv y(n)$ is optimal if changing $y_n$ to $y_n+\epsilon$ doesn't increase profit. That is, $(y_n+\epsilon)-(y_n+\epsilon)^2-(y_{n+1}-y_n-\epsilon)^2-(y_n+\epsilon-y_{n-1})^2$ is maximized at $\epsilon=0$. The first order condition yields $y_n=(1+2y_{n+1}+2y_{n-1})/6$.

Replacing with $\{z_n\}\equiv\{y_n-0.5\}$ results in $z_{n+1}=3z_n-z_{n-1}$. This recurrence relation represents alternate terms in Fibbonacci series:

$F_{n+2}=F_{n+1}+F_{n}=2F_n+F_{n-1}=3F_n-F_{n-2}$.

With variable transformations, we can get $z_n=F_{2(n-1)}z_2-F_{2(n-2)}z_1$ for $n>1$ where $F_0=0,F_1=1$.

Suppose we want to maximize aggregate discounted profit with $\delta$ as discount factor. Aggregate discounted profit equals

$\sum{\delta^n[0.25-\{F_{2(n-1)}z_2-F_{2(n-2)}z_1\}^2]}-\sum{\delta^n\{(F_{2(n-1)}-F_{2(n-2)})z_2-(F_{2(n-2)}-F_{2(n-3)})z_1\}^2}$

Assume $F_{-1}=1$ in above. The first order condition for maximization with respect to $z_2$ is

$z_2=\frac{\sum{\delta^n[F_{2(n-1)}F_{2(n-2)}+\{F_{2(n-1)}-F_{2(n-2)}\}\{F_{2(n-2)}-F_{2(n-3)}\}]}}{\sum{\delta^n[F_{2(n-1)}^2+\{F_{2(n-1)}-F_{2(n-2)}\}^2]}}z_1$.

The optimal ratio $z_2/z_1$ equals $r=1/\phi^2=0.381966...$ where $\phi=1.618034...$ is the Golden ratio. Since $r$ satisfies the equation $r^2=3r-1$, the solution $z_{n+1}=3z_n-z_{n-1}$ simplifies to $z_n=r^{n-1}z_1$. Substituting given initial condition $y_1=0.2$ or equivalently $z_1=-0.3$ gives $y_n=0.5-0.3r^{n-1}$.

Note that we can skip a few steps once we get recurrence equation for $z_n$ by noting that the series should be a geometric series if the solution does not depend on time and all future $z$'s are proportional to $z_1$.