Using the ELO Rating System on Static Objects

223 Views Asked by At

The Setup

Suppose we have a list of movies $m_1, m_2, \dots, m_n$ that we wish to rank in order of "quality." We define the "strength" of a movie $a$ by a function $f$ which takes in numerical parameters (like acting, dialogue, etc. on a scale of, say, 1-10) and outputs some number $f(a)$.

This may already be enough to rank the movies, i.e. if $f(a)>f(b)$ then we can say movie $a>$ movie $b$. However, the function $f$ is self contained and does not compare the movies side by side in any way.

Now, suppose one implements the Elo ranking system to directly compare the movies pairwise. We will take each of the $n \choose 2$ possible comparisons and put them against each other. We will say that if $f(a)>f(b)$, then movie $a$ beats out movie $b$ in the comparison, and both scores will be adjusted in accordance to the ELO system.

Clearly, this is a static comparison as there is no indeterminacy in the outcome (unlike in, say, chess).

The Question

After these $n \choose 2$ comparisons, is it possible for the movies to be ranked in a different order than if we just ranked them according to the function $f$?

If not, what if we instead had some indeterminacy in the outcome of comparisons-- say, movie $a$ beats movie $b$ with a probability of $\frac{f(a)}{f(a)+f(b)}$. This would almost certainly shift the order of the rankings, but would it be "worth it" i.e. would this be an effective means of comparing the movies?