Statistical osu SR & performance calc

[joseph-ireland]

I'm proposing a new SR & difficulty calculation which aims to:

• properly incorporate diff spikes and length into SR - SR now represents difficulty to FC, not peak difficulty.
• give better judgements of imperfect plays by assessing the difficulty to get a given combo and miss count, rather than a flat punishment for every map

This issue is the same as the bottom half of #64 which has become a bit hard to follow.

There's a draft writeup of the methodology here.
The code is at PR ppy/osu#4773 (I've also got an osu-tools branch here with a few improvements)

A few results:

freddie benson
walkingtuna
idke
nathan on osu
karthy

me (~50k)
starbin1 (~100k)

Todo:

• tweak parameters for best results. The most important are StarBonusPerLengthDouble and StarBonusBaseTime in osu.Game.Rulesets.Osu/Difficulty/OsuSkill.cs (they can also be overriden separately in Aim.cs and Speed.cs)
• make abraker95's suggested changes to the write-up
• possibly refactor to simplify the implementation - it's currently transforming strain using log(x) before doing calculations, then transforming back at the end. Following the log(x) through all the calculations simplifies some stuff, adds more concrete meaning to the parameters, and makes the implementation match the writeup more closely.
• Add SR multiplier to control SR inflation

[HeBuwei]

I feel the expected length is a perfect measure for pp but not suitable for SR.

Under the live SR algorithm, if you add an easy part at the beginning of a hard map, the SR basically won't change. This behavior makes perfect sense, and people are already used to it, so it would be good to preserve it. The expected length approach does not have this property.

I personally would choose FC probability per play for SR calculation. It has the aforementioned property, and the way it changes with length is similar to what we have now.

[tr3sleches]

#64 (comment)
This is my response to your last statement in issue #64, just got around to responding.
I agree with the notion that expected length by itself is good for pp, but not SR.

Edit: I just saw a flaw in the integration I just did. It's vastly overstated because properties do not really work since this expected value isn't really calculated properly (which is okay since "proper" doesn't really matter given how it's used right now).
Multiplying the integral part by 2/(t_(n+1) - t_n) for each object n fixes it, but it bugs me XD.

[joseph-ireland]

@HeBuwei I agree, I thought this too, FC probability is the closest analogue to current star rating. I quite like the idea that star rating more closely reflects PP, but I guess that's not what it's for. I was undecided between which one to use, but it was easier to just use the same value as the PP calculations. I haven't actually looked at star ratings in detail yet, I was just focused on PP, so maybe that's something to compare.

[tr3sleches]

Here is your proof for the expected time to FC:

Expected Time to FC proof.docx

In hindsight, I shouldn't even have to do this because all this is a mathy way of saying you retry the map from the beginning every time you miss.
I hate proofs.

[joseph-ireland]

I don't get it. Why number of retries and not number of attempts? The first attempt still takes time. Put in p_i = 1 in your calculations and suddenly everything breaks. Change E[retries] to E[attempts] and it matches what I've got.

[tr3sleches]

If you get the section in 1 attempt, you don’t have to retry the map and the entire
p_i = 1 doesn’t break it.
I formatted it this way for interpretation sake but that 1-p_k section cancels out so it still works.
The first attempt is different from the subsequent ones. If p_i = 1, R_i = 0
Infinity*0 is 0.
It doesn’t break...

[joseph-ireland]

OK, so R_i = 0. Putting that in your first equation gives E[T_n] = delta t_{n-1}, so you expect to FC the map in less time than one play through, which doesn't make sense. It should be E[T_n] = t_n, i.e. sum delta t_i if all the p_i are 1

[joseph-ireland]

Unless you're actually trying to calculate expected time spent in failed attempts before an FC, which actually isn't a bad idea, i.e. E[T_fc] - map_length. That's always going to have a limit of zero as skill goes to infinity, so you don't need to special case long maps, and all the calculations will still work, just need to replace the targe_FC_time with target_FC_time + map_length.

[tr3sleches]

If R_i = 0 for all i, you just add up the deltaT so you just get the map length
Look at the very first equation to visualize this a little more easily.

[tr3sleches]

I put some other ways of expressing the solution:
Hope this clears stuff up.
Expected Time to FC proof.docx

[tr3sleches]

Oh, I see what you mean. I had this fixed when I worked this out on paper, forgot to put it in the word doc.
One sec.

[joseph-ireland]

wait., does E[T] = sum E[T_i]? i was thinking E[T] = E[T_n], so it wasn't making sense. I still don't see how you get to the final answer, seems like it should be more complicated.

edit: went through this with tr3s, the end result is the same as what's already there.