Let's call an ability a pair of two integer numbers: damage and cooldown. We have a bunch ($\ge 8$) of abilities. We must find a set of 8 abilities from that bunch so that average damage per turn will be maximum.
Note that every ability can be used once every $\mathcal N$ turns, where $\mathcal N$ is the cooldown of this ability. Only one ability could be used per turn
If there is a good computing algorithm to solve this problem? Is there some areas of math I should look into to find a clue?
I think the central part of the problem is to find an algorythm to find the best accomodation so that abilities in the set will conflict as little as possible
This feels like an NP-hard problem to me like the knapsack problem. It seems unlikely there is a simple algorithm that handles all the cases if you have a huge pile of abilities to choose from with funny numbers of damage and cooldown. On the other hand, with a reasonably sized pile you can just try all the cases and often many of them will be obviously bad.
Even in the long term case you can't just compute the damage per round for each ability. If you have one that does a lot of damage rarely you are encouraged to choose others that recharge more quickly. Another approach is to have a lot of abilities that all cool down in 8 so you cycle through them. There will probably be a small number of cases to check.
Startup effects can play a role as well. Say you have one with damage of 100 and cooldown 1000. You can only use it once if the battles last less than 1000, but maybe it blows most things away on the first shot. If you can space your battles out enough that it cools down you don't need any others.