Given an $X_1,...,X_n \sim Unif[0,1]$ i.i.d. sample, after ordering them, one of $X_k^*=x$, give a maximum likelihood estimation for $k$ using $x$.

Question

The problem:

The title sums up the problem pretty well, we have a sample from an independent idential (uniform) distribution (i.i.d.):

$X_1, ..., X_n \thicksim Unif[0,1]$,

and we only know one exact value from this: someone told us that "one of the values is equal to $x$":

$X_k^* = x$ for an unknown $k \in \{1,\dots,n\}$, where $X_1^*\le ...\le X_n^*$ is the ordered $X_1, ..., X_n$ sample.

Give a Maximum Likelihood estimation for $k$, using the value of $x$.

Here is what I've done so far:

Let's order the sample into $X_1^*, \dots, X_n^*$. Now the density function of $X_k^*$ follows a Beta-distribution of $X_k^* \thicksim Beta(k,n-k+1)$:

$$\lim_{\varepsilon \rightarrow 0}\Bbb{P}(x - \varepsilon < X_k^* < x + \varepsilon) = f_{X_k^*}(x) = \\ = \frac{\Gamma(n+1)}{\Gamma(k)\Gamma(n-k+1)}x^{k-1}(1-x)^{n-k} = \\ = \frac{n!}{(k-1)!(n-k)!}x^{k-1}(1-x)^{n-k} = \\ = k\binom{n}{k}x^{k-1}(1-x)^{n-k}$$

When given a sample of one $X_k^*$, the Likelihood function is the same as the density function:

$$L_{n,k}(x) = k\binom{n}{k}x^{k-1}(1-x)^{n-k}$$

I set up the log-Likelihood equation:

$$\ln L_{n,k}(x) = \\ = \ln(k) + \ln\left(\binom{n}{k}\right) + (k-1)\ln(x) + (n-k)\ln(1-x)$$

And differentiate it in terms of $k$, because I want to find its maximum in $k$:

$$\frac{d}{dk}\ln L_{n,k}(x) = \\ = \frac{1}{k} + \frac{d}{dk}\ln\left(\binom{n}{k}\right) + \ln(x) - \ln(1-x)$$

I used Wolfram Alpha to calculate $\frac{d}{dk}\ln\left(\binom{n}{k}\right)$, and it turns out to be:

$$\frac{d}{dk}\ln\left(\binom{n}{k}\right) = H_{n-k} - H_k$$

where $H_n$ is the $n$th harmonic number:

$$ H_n = \sum_{i=1}^n \frac{1}{i}$$

So going back to the original equation:

$$\frac{d}{dk}\ln L_{n,k}(x) = \\ = \frac{1}{k} + H_{n-k} - H_k + \ln(x) - \ln(1-x) = \\ = H_{n-k} - H_{k-1} + \ln(x) - \ln(1-x) \stackrel{?}{=} 0$$

Now this is where I'm stuck. I have no clue how to order this equation to $k$ on one side, everything else on the other side; because that is how I would get a maximal value for $k$.

An obvious answer:

The solution would be something like:

$$k = nx$$

Since if we have a sample of size $n$, and $x \in [0,1]$, we'd expect the place of $x$ to fall close to its size (factored up by $n$).

Or to be more specific, $k$ could be:

$$k = (n-1)x+1$$

Since this formula runs from $1$ to $n$, not $0$ to $n$.

But I cannot figure out how we arrive at this. Any help would be appreciated!

Answer 1

You want to find integer value of $k$ that maximizes $L_{n,k}(x)$. To do it, it is sufficient to consider $$\frac{L_{n,k+1}(x)}{L_{n,k}(x)}=\frac{x}{1-x}\cdot\frac{n-k}{k}$$ and compare it with $1$.

We get: $L_{n,k+1}(x)\geq L_{n,k}(x)$ for $k \leq nx$ and $L_{n,k+1}(x)\leq L_{n,k}(x)$ for $k \geq nx$.

So, for $nx\in\mathbb Z$, we have two values of $k$ where $L_{n,k}(x)$ attaines maximum: $k=nx$ and $k=nx+1$.

For $nx\not\in\mathbb Z$, $k=[nx]+1$ is MLE.