Strategy in a waiting time game: fisherman's dilemma

Question

Suppose there is a fisherman out on the lake, who repeatedly casts his line looking for fish. Let $X_k$ denote the time it takes him to catch a fish on his $k$th cast, $k \in \mathbb{N}$, where the $X_k$ are iid, sampled from some distribution function $F$, and so that that $X_k > 0$ and $\mathbb{E}X < \infty$. The fisherman knows $F$, and has control over one thing: he can choose to re-cast his line at any time. What is his optimal strategy to catch the most fish in the long run?

There are situations where it is advantageous to re-cast: if $F$ is bi-modal, with the two modes very far apart, then the fisherman will wait for the first mode to pass, and if he hasn't caught a fish, re-cast the line rather than waiting to hit the next mode.

I can characterize the distribution functions where it is sometimes advantageous to re-cast via a direct computation, though I am curious if there is a more elegant condition. If the fisherman hasn't seen a fish up to time $t$, then his expected waiting time to see a fish is

\begin{equation} \mathbb{E}[X - t | X > t] = \int_{t}^\infty \frac{F(s)}{F(t)} ds, \end{equation}

where I'm using $F(x) = \mathbb{P}(X > x)$. On the other hand, the unconditional expected waiting time is

\begin{equation} \mu = \mathbb{E}[X] = \int_0^\infty F(s) ds. \end{equation}

Thus, it is advantageous to re-cast our line at time $t$ if the conditional expected waiting time is larger than the unconditional waiting time, i.e. any time $t > 0$ satisfying

\begin{equation} F(t) \int_0^\infty F(s) ds < \int_t^\infty F(s) ds. \end{equation}

$\Big($EDIT 2: @DaneiWeissman pointed out that this condition isn't always the right one to figure out the optimal strategy. Assuming that the right strategy is to pick a time $t$ and always re-cast at that time, the waiting time for that strategy is

\begin{equation} \mathbb{E}[X|X \leq t] + \sum_{j \geq 0} F(t)^j(1-F(t)) jt = \frac{1}{1-F(t)} \int_0^t (F(s) - F(t)) ds + \frac{t F(t)}{1-F(t)} \end{equation}

\begin{equation} =\frac{1}{1-F(t)}\int_0^t F(s)ds. \end{equation}

This is because we wait a geometrically-distributed number of times with failure probability $F(t)$ before there is a fish in $[0,t]$, and once this does occur, it takes time $\mathbb{E}[X|X \leq t]$ to catch it on average. So we should be instead be trying to solve

\begin{equation} \frac{1}{F(t)} \int_t^\infty F(s) ds = \frac{1}{1-F(t)} \int_0^t F(s) ds, \hspace{5pt} (\star \star)\end{equation}

or perhaps just minimize the RHS over $t \in \mathbb{R}$. $\Big)$

For example, if these expressions are equal for all $t$, then differentiating with respect to $t$ yields

\begin{equation} \mu F'(t) = -F(t), \end{equation}

and $F(t) = e^{-t/\mu}$ is the only distribution function on $[0,\infty)$ satisfying this ODE. So if the $X$'s are exponential then re-casting at any time never hurts or helps us. (This shouldn't surprise anyone familiar with Poisson processes, and the memory-less property of the exponential distribution!)

One can also compute directly with $F_\alpha(x) = (1+x)^{-\alpha}$, for $\alpha > 1$. This gives

\begin{equation} \mathbb{E}[X - t | X > t] = \frac{t+1}{\alpha - 1}, \end{equation}

while

\begin{equation} \mathbb{E}X = \frac{1}{\alpha - 1}. \end{equation}

So the fisherman should recast his line at every positive time in this case!

I am interested in the times $t$ where we have equality, namely times $t$ when

\begin{equation} \mathbb{E}[X-t|X > t] = \mathbb{E}[X] \hspace{10pt} (\star) \end{equation}

My questions are:

1) What possible sets can occur as solution sets to $(\star)$ or $(\star \star)$? For example, can there be a single unique solution? $N$ solutions for some $N \in \mathbb{N}$? Can it be satisfied for all $t$ in some interval?

2) Is there a nicer condition to determine whether or not there is some solution to $(\star$) or $(\star \star)$? (I wonder if there is some way to compare $F$ to an exponential distribution that would help.)

3) Can the integral equations $(\star)$ or $(\star \star)$ be reduced to a more tractable form?

4) Are there any other nice classes of distribution functions for which they can be solved explicitly?

5) How would one make sense of the case where $\mathbb{E}X = \infty$? The equations $(\star)$ and $(\star \star)$ don't make sense anymore. Should the fisherman ever re-cast in this case?

Also, I am curious if this question has been studied in any other context: it seems like a very natural setup to me!

P.S. There are many simple possible variants on the original question. I wonder what would happen if one imposed a time-cost on re-casting the line, or if the fisherman doesn't know how long has passed since he cast his line. What would the optimal strategy be?

EDIT 1: Another distribution one can prove something about is $F(t) = e^{-t^2}$, which decays faster than exponentially. We have $\mathbb{E}X = \frac{\sqrt{\pi}}{2}$, while

\begin{equation} \mathbb{E}[X-t | X > t] = e^{t^2} \int_{t}^\infty e^{-s^2} ds. \end{equation}

It is straightforward to check that this function is strictly decreasing, and

\begin{equation} \lim_{t \to 0^+} \mathbb{E}[X-t | X > t] = \mathbb{E}X, \end{equation}

which holds for any distribution function $F$ (since $F$ has right limits and $F(0) = 1$). Thus, in this case the fisherman should never recast his line.

Answer 1

Let $f$ be the probability density function, $t$ be the recast time and $T$ be the resulting average catch time. We have: $$ \begin{align} T &= \int_0^t x f(x)\ \mathrm{d}x + \left(\int_t^\infty f(x)\ \mathrm{d}x \right)(t + T) \\ T &= \frac{\displaystyle \int_0^t x f(x)\ \mathrm{d}x + t \int_t^\infty f(x)\ \mathrm{d}x}{\displaystyle 1 - \int_t^\infty f(x)\ \mathrm{d}x} \label{1}\tag{1} \end{align} $$ The minimum is either at $t = 0$ (recast as soon as you can), $t = \infty$ (never recast), at $\frac{\mathrm{d}T}{\mathrm{d}t} = 0$ or at a discontinuity of $f$. The case $t = \infty$ is simply the expected value. For $t = 0$ we take the limit to 0 and need to use L'Hôpital's rule. $$ \lim_{t \downarrow 0}\ T = \lim_{t \downarrow 0}\ \frac{\displaystyle \int_t^\infty f(x)\ \mathrm{d}x}{f(t)} = \frac{1}{f(0)} $$ Now the case $\frac{\mathrm{d}T}{\mathrm{d}t} = 0$: $$ \begin{align} \frac{\mathrm{d}T}{\mathrm{d}t} &= \frac{ \left( \int_t^\infty f(x)\ \mathrm{d}x \right) \left( 1 - \int_t^\infty f(x)\ \mathrm{d}x \right) - \left( \int_0^t x f(x)\ \mathrm{d}x + t \int_t^\infty f(x)\ \mathrm{d}x \right) f(t) }{ \left( 1 - \int_t^\infty f(x)\ \mathrm{d}x \right)^2 } \\ 0 &= \left( \int_t^\infty f(x)\ \mathrm{d}x \right) \left( 1 - t f(t) - \int_t^\infty f(x)\ \mathrm{d}x \right) - f(t) \int_0^t x f(x)\ \mathrm{d}x \end{align} $$ Some examples:

• The exponential distribution $f(x) = \lambda e^{-\lambda x}$ has $T = \frac{1}{\lambda}$. It's the only distribution in which $T$ does not depend on $t$.
• In a delayed exponential distribution $t = \infty$ gives the minimum value for $T$. $$ f(x) = \begin{cases} e^{1-x} & x \ge 1 \\ 0 & x < 1 \end{cases} $$
• With the following distribution the minimum is at $t = 1$. $$ f(x) = \begin{cases} \frac{1}{2} & x \in [0,1] \vee x \in [3,4] \\ 0 & \text{otherwise} \end{cases} $$
• This distribution has minima at $t = 1$ and $t = \frac52$: $$ f(x) = \begin{cases} \frac{1}{3} & x \in [0,1] \vee x \in [\frac32, \frac52] \vee x \in [5, 6] \\ 0 & \text{otherwise} \end{cases} $$
• This gives a minimum at any $t \in [0, 1]$: $$ f(x) = \begin{cases} e^{-x} & x \in [0,1] \\ \frac{1}{e} & x \in [2,3] \\ 0 & \text{otherwise} \end{cases} $$
• This gives a minimum at $t = 0$: $$ f(x) = \begin{cases} e^{-x}-\frac{1}{e} & x \in [0,1] \\ \frac{2}{e} & x \in [2,3] \\ 0 & \text{otherwise} \end{cases} $$
• The following distribution has expected value $\infty$ and has a minimum at $t = 0$. $$ f(x) = \frac{1}{(x+1)^2} $$

To your question if the fisherman should ever recast if the expected value of the distribution is $\infty$: Yes, he should always do that. Recasting gives a finite value for $T$ while not recasting will give $T = \infty$.