I have been trying to come up with a formula that would allow ranking competitors according to their success at solving certain tasks, but with a number of criteria in mind. Maybe something exists that I don't know of. Here is the description of what I'm trying to do. Think of it as a competition where each participant can create new tasks for others to solve.
- Tasks have different difficulties and therefore earn solvers different number of points
- Participants can create new tasks of varying difficulties if they want to.
- Participants aren't allowed to solve the tasks created by themselves but should not be penalised in the final ranking.
- Contributing by creating a task should not have any positive or negative impact on their rank.
- It is necessary to fairly calculate ranking for someone who creates a very hard task vs someone that creates an easy task.
What I was thinking of doing is get rid of the concept of points and replace it by "percentage of completion". But then how to take into account tasks difficulties? I thought about thinking of difficulty as a % multiplier and normalise according to the number of tasks the participant is allowed to solved. For each participant, getting 100% completion would mean solving all tasks except their own. But I guess it creates a big flaw: what if a participant creates several hard tasks that nobody manages to solve? Could they win the competition by solving a few easy tasks and creating a lot of hard ones?
I'm really struggling here and not sure that it's even doable. Help would be much appreciated!