Let's say that I organize a week of sporting events. Each event is considered equally difficult --- winning in one event should be just as rewarding as winning in any other event.
Each athlete has the ability to participate in all events, but they might not. If they participate, then they finish, and get a score which indicate their performance. That is to say, for a 60 meter run, they get a score of 9 seconds, or 11 seconds, or whatever, not just a ranking of who came 1st and 2nd.
The performance metric matters, not just the ranking of coming 1st or 154th. If you run much faster than you competitors, that should reward more than just slightly finishing ahead.
The scoring metrics of each event is different. While each event is equally important, the metrics are different. The metrics are not directly comparable across events. Running a 9 second 60 meter sprint is impressive, but needing 9 tries to hit a dart board is not so good.
I'm looking for ways to score each participant for their total performance in this multi-sport Olympics.
There is an assumption that needs to be made here. How to handle the score of a non-participant? What options to I have here?
I'm looking for insights into some data, and both extremes would yield me that. Thus I'm looking for two different scorings, for each participant:
The first should penalize non-participation. It would rank highly those who participate in many events and who score decently in each of them. That is, all-rounders would be highly ranked.
The second should not penalize non-participation that much. If you always out-perform all competitors by a large margin, but only participate in a few events, then you would score highly.