Estimate or estimator in KDE

Question

I have a question about when one uses "estimate" and "estimator" in the context of kernel density estimation.

I read that ''an estimator is a function of the observations, a specific way of putting them together. It may be specified by an arithmetic formula , like $\bar y = \sum_{i=1}^n x_i/n$, or by words alone, as in directions for finding the sample median by ordering and counting. We distinguish between the estimator and its value, an estimate, obtained from a specific set of data. The variance estimator, $s^2 = \sum_i(x_i-\bar x)^2/(n-1)$, yields the estimate $7$ from the three observations $2, 3, 7$. We say $s^2$ is an estimator for $\sigma^2$, and we call $\sigma^2$ the estimand. In the numerical example, $7$ estimates $\sigma^2$.'' (From Jones 1987, The collected works of John Tukey, page 633).

So if I have some random variables $X_1,...,X_n$ and I define the kernel density estimator $$f(x) = \frac 1 {nh_n} \sum_{j=1}^n K\left(\frac{x-X_j}{h_n}\right),$$ is it really an "estimator" or an "estimate"?

Because in the literature that I have read, they always call it the kernel density "estimate". But by the definition above I would have thought that it is an estimator, since I am considering random variables, which have not yet been assigned a value; and then only when I have a certain dataset, i.e. $X_1 = x_1,...,X_n = x_n$, then the value $$f(x) = \frac{1}{nh_n}\sum_{j=1}^n K\left(\frac{x-x_j}{h_n}\right)$$ is the estimate.

Am I understanding the definition in Jones incorrectly?

I would greatly appreciate it if someone could help clear up my confusion. :-)

Answer 1

Graphical comment. I'm not sure about the context of your question or about why you you are looking at KDEs.

If you don't know about the density procedure in R, you may be interested to know that it fits KDEs to data. In a sense, you can view KDEs as "smoothed histograms" in which the bandwidth of the KDE is somewhat analogous to the width of the histogram bins.

Consider a sample of size $n = 100$ from $\mathsf{Norm}(\mu=100, \sigma=15).$ Below we show a histogram of the data with a plot (dotted) of the density function of the population distribution, and also a plot (solid) of the default KDE in R. The KDE is based only on the individual data values (shown as tick marks of the horizontal axis), and is made independently of the histogram.

set.seed(2020)
x = rnorm(100, 100, 15)
hist(x, prob=T, col="skyblue2", main="n=100");  rug(x)
 curve(dnorm(x, 100, 15), add=T, lwd=2, lty="dotted")
 lines(density(x), lwd=2, col="red")

Here is a similar situation, but with a sample of size $n=10\,000.$ (Tick marks are omitted here because there are too many to of them to show individually. I have put about twice the number of bins in the histogram, but I'm still using the default bandwidth of R.) For this larger sample, the population density curve and the KDE are very nearly the same.

set.seed(904)
x = rnorm(10000, 100, 15)
hist(x, prob=T, br=25, ylim=c(0,.03), col="skyblue2", main="n=10,000")
 curve(dnorm(x, 100, 15), add=T, lwd=2, lty="dotted")
 lines(density(x), lwd=2, col="red")

You can look at the documentation on R's density procedure to see how to change the type of KDE and its bandwidth.