-
Notifications
You must be signed in to change notification settings - Fork 18
Rating vs Ranking
Dev Ccnference #7 https://youtu.be/y_Sm5-uQkHA?t=1h7m50s
Dev Conference #8 https://youtu.be/0q0SFU6Mt0M
Dev Conference #9 https://youtu.be/RelvYYh4z3I
https://faforever.slack.com/archives/C02T320UJ/p1493122495272543
jackherer [14:14] Last news is messing up with rank and rating ...
mazornoob [14:15] ?
jackherer [14:15] EQ becoming ranked is wrong
[14:15] EQ becoming rated
[14:16] there is no rank in glabal
[14:16] global
speed2 [14:16] EQ is rated, affecting only global rating
keyblue [14:17] what do you call this though? https://www.faforever.com/competitive/leaderboards/global Isn't that a ranking system?
jackherer [14:17] yea why do we use the word rank everywere on the news
speed2 [14:17] cause noobs
jackherer [14:18] there is no rank on global keyblue
[14:18] and we use the word rank at 6 time on last news
[14:19] who made the news? is the council also confuse between rank and rated?
downlord [14:20] yes, obviously. I'm with jackherer on this
icedreamer [14:20] There's no confusion with the devs, just that we don't HAVE a rank system, so the ratings default to being a rank. It sucks
duk3luk3 [14:20] Because "it's ranked" is what everyone calls it
downlord [14:21] that's not true. being 1st is a rank, having a rating of 2600 isn't a rank
jjs_ai [14:21] @jackherer I assume you're talking about the recent developer conference post?
duk3luk3 [14:21] Everyone in faf calls it that.
[14:23] Games and mods are either ranked or unranked
[14:23] It's always been called that
jackherer [14:23] the rating is a tool to balance
[14:23] and btw trueskill shouldnt be display with a number
[14:23] it whats make the confusion with a rank
jackherer [14:30] everyone is wrong, and it's what make custom teamgame that should be fun and easy to play with your friend a pain in the ass
icedreamer [14:31] We have a solution
jackherer [14:31] so yea we miss a competitiv mod for teamgame but still we can't mess and confusing more than it is
icedreamer [14:31] But it needs a lot of coding
jackherer [14:32] i will change rank with rated in the french translation.
[14:33] what's your solution icedreamer?
downlord [14:34] oh, just because people have been calling it wrong, let's continue to spread the wrong term. great idea.
icedreamer [14:35] The plan
dragonfire [14:35] One stupid question... Can we rename rank/rating? Maybe to performance... Player level... Skill level... League...
icedreamer [14:35] Is to eventually hide the rating entirely, behind a proper division-based ranking system
dragonfire [14:35] Maybe people mix rating and ranking
icedreamer [14:36] Each rank will contain people of, for example, 200 rating points. So take 1000 - 1200 as one bracket, for example
[14:36] it is likely we will also make it slightly easier to climb than to fall. So going from 1198 to 1200 will put you into the next bracket
[14:37] But you'll need to fall to 1150 to come down.
dragonfire [14:37] I think that was sheeo plan... And then call it silver league, gold league... Or silver bracket.... Silver level.... Silver Tech... Silver area...
icedreamer [14:37] Each bracket has a name reflecting their relative skill, similar to any other game right now
jackherer [14:37] Yea use trueskill to give grade and hide the number. Like 1000-1200 you are a wagner 1200-1400 you are a titan ....
icedreamer [14:37] novice, initiate, veteran, avatar of war, etc
jackherer [14:37] yea
[14:37] i wanted that since so long 😄
icedreamer [14:37] Ratings will be visible to yourself with right-click detail options somewhere, but likely not to others
[14:38] And rating of course still used to calculate matchups
[14:38] BUT AS I SAID
jackherer [14:38] and just balance by puting a caporal vs a caporal
dragonfire [14:38] Can somebody please create an issue for this that we have this in stone and sheeo can approve this?
icedreamer [14:38] There is a LOT of coding to make this happen
[14:38] it's a long ways off
jackherer [14:38] it's a perfect solution
speed2 [14:38] stop teasing
sheeo [14:38] … No, it’s not hard to do
speed2 [14:38] no one is coding that 😞
sheeo [14:38] And no, it’s not a perfect solution
dragonfire [14:38] Agree
icedreamer [14:38] This was already spoken about several dev conferences ago
dragonfire [14:38] Specially with the new api...
speed2 [14:38] but ppl been talking about it for years
sheeo [14:38] At least not what Jackherer described just above
icedreamer [14:38] and as I remember, agreed upon
sheeo [14:39] You can’t balance from a division of peoples rankings, that undermines the entire point of trueskill
jackherer [14:39] it's already better than messing with trueskill number, that make everyone even the council being confuse between rank and rate
icedreamer [14:39] sheeo you matchmake on rating
duk3luk3 [14:39] jackherer you're majorly pissing me off here
icedreamer [14:39] of course. Just you display the division, exactly like other games do with MMR
dragonfire [14:39] Can anybody link me to the dev conferences?
sheeo [14:39] jackherer just said “just balance by puting a caporal vs a caporal”
[14:39] No.
icedreamer [14:39] oh right
jackherer [14:39] why not?
icedreamer [14:39] Yeah no we don't do that
[14:40] Because sometimes that won't be the correct matchup
sheeo [14:40] Because then you’re just ignoring all of the information behind the rating
jackherer [14:40] trueskill is so biass depending of map spot ...
sheeo [14:40] And assuming that everyone is performing at mu-3*sigma
icedreamer [14:40] For example, if there are very few people seeking a match, or if someone's rating has a very high dev. Right sheeo?
sheeo [14:40] Ignore any bias for now and just assume perfect, identical conditions
jackherer [14:40] it has never been
sheeo [14:40] You’re not going to get anything straight without a system that gets at least ideal conditions right
jackherer [14:40] and will never been perfect
icedreamer [14:41] There's also something to be said about: Do we use mu-3*sigma, or mu, to denote bracket transitions
sheeo [14:41] Your idea will match people who we are very confident will play as-a 1205 against people whom we are very confident will play as-a 1399
[14:41] E.g. terrible balance
jackherer [14:42] tbh sheeo a 94% game can be a terrible balance if the host know the player and pick the op one
sheeo [14:43] Pick the op one?
[14:43] You mean a teamgame?
[14:43] You didn’t even provide a matchmaking algorithm for teamgames
[14:44] Regardless; divisions should not be based directly on trueskill brackets either, @icedreamer
jackherer [14:44] yea i mean teamgame
sheeo [14:45] Yes, trueskill game quality estimates cannot be reliable when the host intentionally games the teams
[14:45] This should be obvious.
jackherer [14:46] so i do think that's it's more important to make it easier to fill teamgame even if balance is less accurate
[14:46] because it wil never been accurate
philipjfry [14:46] the wrong wording on the what's new post is my fault
[14:46] if anyone wants to change it be my guest
[14:46] the draft was up for everyone to see and suggest changes for 2 days btw
sheeo [14:46] You’re conflating matchmaking with teamgame filling
[14:46] These are inherently unrelated
jackherer [14:47] i am not talking about matchmaking
[14:47] sheeo
[14:48] i'm only talking of custom teamgame, and the fact that using trueskill with a number confuse everyone about rate/rank
sheeo [14:48] Using a bracketed division system will not make this less confusing
jackherer [14:48] ofc
[14:48] it will
sheeo [14:48] Your argument still holds — the host can have information that some player is over/undervalued and place them according to this information
icedreamer [14:49] You can't stop players gaming the system, ever
sheeo [14:49] Indeed.
philipjfry [14:49] you'd have to obfuscate the number so much that no information can be derived from whatever we show to the player to make it impossible for a host to stack
icedreamer [14:49] Just like a team of 4 1200s on mic will usually beat a team of 4 1400s who don't communicate
[14:50] There are variables in play aside from raw skill
sheeo [14:50] Yes
[14:50] Regardless, @icedreamer — what kind of divisioning were you thinking?
philipjfry [14:51] @dragonfire a link to the video is in the latest what's new post 6 replies Last reply today at 15:13 View thread
sheeo [14:54] https://www.youtube.com/playlist?list=PL0nxuIUIjpFvX-AFE5Qm0QUuC8gtaJ7yU — I updated the playlist
sheeo [14:54] Please don’t use threads, they’re cancerous 😛 1 reply Today at 14:57 View thread
icedreamer [14:55] Well, ~200 rating gap seems to me like a good 'bracket' size
jackherer [14:56] yes it is most of the time, depending of map spot premade, a 1400 can face a 1600 or a 800 can face a 1000
icedreamer [14:56] Have the system prefer to match within a bracket if it can, but then fall back on pure TS if needed. Other games do this too, as it means that a player who's newly gained a new higher bracket is once again being pushed by slightly better players
[14:56] Though I'm definitely thinking about ladder here
[14:57] not teamgames
sheeo [14:57] So we are talking about changing matchmaking
[14:57] I don’t see why this needs changing at all, especially 1v1 ladder
icedreamer [14:57] It all becomes soooo much harder in teamgames, if not impossible
philipjfry [14:57] ladder doesn't have that many issues with rating imho
jackherer [14:57] yea i think ladder isn't an issue
icedreamer [14:57] Weren't you at that conference mate? We don't have a ladder right now, really. No climbing, no real rank at least not functional
sheeo [14:57] It already does a sort of implicit bracketing — it’ll expand the search as it can’t find matches
icedreamer [14:57] People want this
[14:57] I know
[14:58] It's about presentation. This whole debate is about presentation
[14:58] how the community see it
sheeo [14:58] Then don’t talk about how it’ll match — that’s logic, not presentation. (edited)
icedreamer [14:58] and use it
jackherer [14:59] to me the issue is that there is no non-competitive way to play on faf. Because or rating being use like a rank. We need a place were we can have fun game and chill, just get the fun side of this game
icedreamer [14:59] Well to start with I would have expected other systems to purely use their TS or MMR or whatever to match players, but looking into it I found that they actually do weight matching within a bracket, as it fosters competitive spirit. People can never reach an easy, calm state, they're always pushed to get better and better
philipjfry [14:59] this is impossible by definition @jackherer
icedreamer [14:59] jack, competitive vs noncompetitive is a mindset, not a game mode
[14:59] I don't play competitively, ever
sheeo [14:59] What kind of other systems are you talking about?
[15:00] LoL, SC2 etc.? (edited)
icedreamer [15:00] DOTA, LOL, and CS:GO all do this, yes
philipjfry [15:00] we either throw all rating out of the window or people will have to deal with losing precious points
sheeo [15:00] All these have a key thing: Large playerbases.
icedreamer [15:00] This is true. Not sure how that'll factor in
sheeo [15:00] We cannot bracket and exclude players from matching, we don’t have this luxury.
icedreamer [15:00] Not saying we exclude, nevter said that
sheeo [15:00] But again, you’re mixing up presentation and logic?
icedreamer [15:00] I said prefer
sheeo [15:00] We already do this.
[15:01] Just, implicitly
icedreamer [15:01] Sort of... Lets do an example
[15:01] Right now, if an 1100, a 1200, and a 1370 are searching, who is most likely to get matched sheeo?
[15:01] I'd imagine the 1100 and the 1200, right?
sheeo [15:01] Will depend on their deviations.
icedreamer [15:02] Lets say their devs are all fairly tight
sheeo [15:02] Then it depends on who started searching first
icedreamer [15:02] these are regular players
[15:02] All searching same time, roughly
sheeo [15:02] There’s no such thing
philipjfry [15:02] the ones who start first find each other
icedreamer [15:02] Ah, we do a strict first-come within a reasonable match hmm
[15:02] OK
[15:03] 1100, then 1370, then 1200 last
sheeo [15:03] Our server is single-threaded and acts atomically for matching (edited)
philipjfry [15:03] 1100 and 1370 will find each other
sheeo [15:03] Even if it wasn’t single-threaded, you’d still have to do an ordering based on insertion
[15:03] Well, iff their deviations allow an 85% quality match instantly, yes
[15:04] Otherwise: no
icedreamer [15:04] Would they, phil? That's a big skill gap, I'd hope that the first rating search would be trying not to
philipjfry [15:04] pretty sure those 2 are within 85% game quality
sheeo [15:04] You cannot say that — it will depend on their deviation.
philipjfry [15:04] ok i assumed that much
[15:04] if they aren't then 1200 will find 1100 first
[15:04] i don't see the issue then tho
sheeo [15:04] What we need people to understand is that your rating is not a number, but a probability distribution
icedreamer [15:05] Well, lets imagine a situation where the 1100 and the 1370 have 50%, and the 1200 and the 1370 have 80%, and the 1100 and 1200 have 95%
philipjfry [15:05] that's how it is supposed to work
icedreamer [15:05] There's an argument to be made to match the 1200 and the 1370, despite the slightly worse matchup
sheeo [15:05] Game quality is the area of the overlap between your skill probability distribution and the one of your matchup
philipjfry [15:05] the playerbase is too small for such a choice
jackherer [15:06] added this Plain Text snippet So i'll try to give you my feeling about whats wrong with how trueskill is exploited on faf global rating Because i think it's a major issue at faf expension. I dont have a magic solution but i really think global rating need a deeper change than multiple rating. So i hope you read it, make your minds and think about something that could be better. I'll start with a little reminder, global rating for custom game set up by zep was initialy a tool for balance and not a rank. I play faf since gpg close (dont even remember how many year it makes) and rating have always been a matter for a lot of people.
- i have many friends who never tryhard on faf because it was too hard for newcomers.
- i know ppl who have quit faf and fa because rating was too much stressfull
- i know ppl who play on steam because they only see faf as a competitiv way of playing
- we all know ppl who cheat on global rating, because they believed it was a rank and try to achieved to be higher than they should.
- we all know ppl that are prisonner of their rating and to scared to play an other map or an other spot because they know they will be abuse. So Lets start with the beginning, new players...
A new player joining faf can have very hard time because of global rating. First of all, he have to get familiar with rating. What are those 1k+ game? Why i am kick from the majority of game? Until he find someone kind enough to explain him that the number near his name is his rating and he must go in game thats fit his rating. Then every newcomers who pass step 1 of understanding rating wont support the fact of having negative rating. We are simply saying them "you suck so much that you are less than 0" Then they will have some failure in game and they will get abuse by player who think the rating is a rank (you know the ppl who reconnect after each game to see their "rank improvement") So i think we loose a huge % of new fafer that are discouraging before they know well enaugh the game. Lets moove on with regular player I'm running the BFA teamspeak server since pretty long now,and i've seen and listen to a lot of ppl there, and i can assure you that after playing teamgame the first reaction is generally like: "omg we are going to loose a lot of point" "damn if i loose one more i'm going to go down 1400" "oh yea it was an 80% game this is going to be a lot of pts" "oh no i'm going to get too high dont want to be 1.8k need to loose pts again" and in game it look like "omg his he a noob how did he get his point" "oh his connexion die free points" "be carefull he is overated/underated put him in the other team" The faf world in custom teamgame is about point. You know that number near your name that zep implement originally to help balance game. So yes balance is important to have good game, i dont deny that. But how many ppl are overated? how many are underated? how many ppl had their rating from ladder before the rating get split? how many ppl are 1500 on a single map and a single configuration because they are too scared to try something else and be abuse? Trueskill for a teamgame RTS can't be perfect, it can be a tool, but here it's more than a tool, its faf god. It remove the possibility of mix social classes, noob only play with noob, casual player play togeather good player only play between them. Few day ago sheppy came with a newcomer in one of our game, we think of the better possible balance to have a fun game and we had a decent game, but i can assure you that in the majority of the game, he wouldn't have been able to play a teamgame with his newcomer friend. I'd like to finish with modded game, few month after gpgnet close, some top player get angry at ppl having getting high rating as well, but playing different kind of game... so zep remove rating for non classic game ...... By doing that i think he killed fa diversity who is (was one of the 1st strength of this game), since this day it became harder and harder to get a non classic custom game. So player playing thermo no rush blackops phantom dont have the right to balance their game, dont have the right to have some reward for their commitment to that game, they dont even have the right to see their number of game increase. Now that i have point the issue, i will explain what i think could be a solution (maybe i'm wrong and you can work on a better one)
- Keep using trueskill but as a tool, to make real grade
- Rate everygame thermo phantom no rush bo .........
- Less grade (today we can have approximativly 25 grade (-500 to 2000) I think 6 or 7 grade could be enaugh 0-500 / 501-900 / 901-1300 / 1301- 1700 /1701-2000 / 2k+
- Make grade being playful and replace the number by icons (i didnt think of the name but 0-500 could be scout 501-900 (mantis) ..... I think this will be way more accessible for newcomer and occasional gamer, will encourage mixing level skill, will make it easier to play with your friend. Will make the balance a bit less accurate, but as said it's already non accurate for teamgame RTS and honestly a 1.7k player can face a 1.9k player a 1.3k can face a 1.6k .... Most of the time
For the real competitive player who wants to excel at classic game, they have ladder, we could make a championship with tournament 2vs2 3vs3 4vs4. Voila, i hope it wasn't tltr, it was just an opinion of an old faf player who thinks that this community could have grown better and could have being more peacefull. If you wanna change things and re-think the balance/rating possibilites i'll be happy to help and think a bit more of how we can do things better. Add Comment Collapse
philipjfry [15:06] letting the 1100 and the 1200 play instead of the 1200+1370 is ok
jackherer [15:06] This a text i start writing month ago about rating
[15:06] and i've never posted because of too much drama on forums
sheeo [15:06] Rating below zero doesn’t make sense
[15:06] Neither does rating above 3k
[15:07] We need to clamp the displayed ratings to this interval everywhere
[15:08] I do realize and agree that showing the rating as prominently as we are leads people to fight over them as if they were “points”
[15:08] I don’t agree with hiding it entirely just for that reason, though.
philipjfry [15:09] lets show them pretty pictures instead of numbers
sheeo [15:09] Just replace it with something like a bracket, sure, but keep the real numbers visible and queryable for people interested in statistics
philipjfry [15:09] yes
[15:10] you can choose a faction and you get put into brackets called like the units are
[15:10] eg a new player goes for cybran and is now a mole
[15:10] instead of an ugly 0
sheeo [15:11] Sounds fine to me
philipjfry [15:11] the system behind it is the same but ppl get less worked up
sheeo [15:11] The reason I don’t think we should base those brackets directly on trueskill, though, is the lack of progression
[15:11] Assume you division the [0;3000] interval in 200-segment brackets
[15:12] A lot of players will have 800-1k, 1k-1.2k, 1.2k-1.4k
[15:12] The rest will be much rarer
[15:12] And you will not progress much over time
icedreamer [15:12] Yes I'd been wondering about that
[15:12] Have not been able to think of a sensible way to do it, I don't know enough about the curve
[15:13] Ideas sheeo?
dragonfire [15:13] maybe downlord achieve system can solve the long time motivation 😉
speed2 [15:13] but at some point you stop progressing no matter the system
[15:13] if you're not playing better how can you get better division?
[15:13] unless its based on how much you play
sheeo [15:13] I just think a more fine-grained division system would be better
[15:13] And perhaps normalized according to the actual distribution
philipjfry [15:14] The distribution doesn't have to be even
sheeo [15:14] E.g. less distance between brackets in the high-population range
[15:14] And longer distance in the low-population range
dragonfire [15:14] or make an exception in the high ranked
[15:14]
if you are at supreme commander level you see a number
icedreamer [15:14] Perhaps bracket it into % of rating populating?
[15:15] So bottom 10% of players, next 10%, next 10% ?
dragonfire [15:15] Tech 1, Tech 2, Tech3, ... Tech 99 (edited)
sheeo [15:15] Well, we should be able to assume the rating population is normally distributed
[15:15] Not quite the truth, but, close enough
icedreamer [15:15] That way each bracket always has roughly the same number of players
[15:15] (It's easier to go up than down these brackets) (edited)
sheeo [15:16] Yes
[15:17] Ok--task force time
[15:17] Come up with division names, say 5 divisions for each faction?
[15:17] And we can sub-divide them into say 2-3 stars for each division if needed (edited)
icedreamer [15:17] By faction hmmm 😕
sheeo [15:17] I do like that idea 🙂
icedreamer [15:17] Theme is good
[15:18] But there names should also be used in global (Even though it's not really a ladder there)
sheeo [15:18] I think as long as there’s a distinguishable theme going on with the icons that’s fine
[15:19] E.g. you can, from looking at a tier-3 aeon icon and a tier-3 cybran one, realize they’re the same division (edited)
icedreamer [15:19] That would work
speed2 [15:21] 5 stars as there are 5 vet ranks?
sheeo [15:22] Actually, 6 divisions is easiest
[15:22] Then we can just define them as this: https://upload.wikimedia.org/wikipedia/commons/thumb/8/8c/Standard_deviation_diagram.svg/400px-Standard_deviation_diagram.svg.png (11KB)
speed2 [15:22] but we have just 5 vets in game 😛
philipjfry [15:22] 0 vet is there too
speed2 [15:23] true dat
sheeo [15:23] Oh, wait, we don’t want that — that’s exactly what we don’t want 😛 (edited)
philipjfry [15:23] Checkmate atheist
sheeo [15:23] Somebody subdivide the normal distribution s.t. the area of each division is equal
[15:26] Regardless, I think it’s easier with a decoupled division system
[15:26] A scheme like this will be prone to sporadic promotion/demotion around the bracket borders (edited)
[15:28] Something like — initial placement into Tier N with 0 “points”. Every game you’re awarded points according to your performance relative to the expected performance (computed using your underlying trueskill)
[15:28] Once you reach, say 5 points, you’re eligible for promotion into the next tier (Or subtier)
[15:29] Once you reach 10 points that happens automatically and you start over in the next tier
[15:30] Losing while at 0 (or below some other number) can demote you
[15:32] @icedreamer if you don’t have a buffering-scheme like this, if you’re right at the border of a division you can be Tier 4 one game and Tier 3 the next, Tier 4 -> 3 ad nauseam. (edited)
icedreamer [15:34] There definitely needs to be a buffer
[15:35] When I was thinking in terms of pure rating brackets, the example I used was that if the border is 1200, then at 1200 you jump up, but to go back down you must sink to 1150
[15:35] Something along those lines. Need to both smooth out transitions to prevent the hopping you describe, AND promote progression by making progression a little easier than regression
sheeo [15:36] We shouldn’t be modifying the trueskill values themselves though — just let trueskill do what it does
[15:36] Adjust this layer built on top accordingly
[15:36] So, when you’re eligible for promotion — you may jump one or more divisons
icedreamer [15:36] I never suggested modifying the TS values 🙂
sheeo [15:36] And when you’re eligible for demotion, you may jump one or more divisions
icedreamer [15:36] That all goes on under the skin
sheeo [15:37] So your actual division can vary a bit (Say 1-4 divisions) from the underlying trueskill-dictated one
icedreamer [15:37] I can't think of a time someone would need to jump multiple divisions. That would suggest divisions are too narrow
[15:37] Nobody ever improves in skill by that kind of degree in a short period of time
sheeo [15:37] Mostly during initial placement
[15:37] And if you’ve been away for a long time
[15:38] Regardless; this isn’t that much coding.
icedreamer [15:39] more than 0 is a lot for this project 😛
sheeo [15:40] Let’s discuss it thursday