Valve’s new and improved system comes with new and improved concerns
I have previously written at length about problems with transparency and clarity in Valve’s Dota 2 system so as you might imagine I am thrilled to see they have finally committed to a much more transparent system.
That said, I think there are still some very significant issues that need to be addressed. The reason I say ‘more transparent’ as opposed to just ‘transparent’ is that there are still areas where transparency is lacking. This is the first set of issues I’ll address.
Direct invites to third party events
It’s always been the case that third party events have some fairly arbitrary power over which teams ultimately qualify for Valve events. This is because they get to decide on direct invites without having to justify them to anyone. Naturally, teams who attend more events have a better chance of proving themselves worthy of invites.
Sadly, the new system actually exacerbates this problem. The greatest irony here is that this is a problem that Valve’s own discretion actually helped to combat in the previous system. They used to be able to look for problems like this and consider them when selecting invited teams. But now that the TI invite system is entirely automated, third party organisers have a much more direct influence over who does and doesn’t qualify. Any event organiser can look at the current point standings, calculate which teams need how many points, and based on that information deliberately invite those teams whom they might want to help qualify for TI. For example, we don’t want a situation where Na’Vi and VP are automatically at every Epicenter or LGD and iG are automatically at every ACE sanctioned event.
This is a specific problem based on ulterior motives and collusion but there is also a more general version of it. The simple fact is that attending more events means you can earn more points and earning more points means you’re more likely to make it to TI. So if certain teams are getting direct invites to third party events it is fundamentally necessary for those invites to themselves be based on transparent criteria. Otherwise the TI system is still not transparent, albeit one step removed. You might suggest that Major invites should be based on performances at Minors. But consider then how Minor invites will be determined and the same issue arises.
Solution: To solve this problem Valve need to insist on certain criteria from third party organisers when determining direct invites. So far they’ve said that at least one qualifier per the main six regions is required. But this doesn’t speak towards how the other slots work. The most obvious suggestion is to make invites to Minors and Majors also based on the same point system. But this is a problem because it creates a serious barrier to entry and will lead to the top achieving teams having an unreasonable advantage, likely to stay inside the circuit indefinitely.
What other options are there? I’m actually not sure. Perhaps using external systems that consider even events that aren’t Minors or Majors would help. One example would be Noxville’s rating system. Perhaps a combination of factors would be ideal. This might be a challenging problem to solve but what is very clear is that leaving the decision to the discretion of third party organisers simply recreates the transparency problem at a different place.
The simplest solution, of course, would be to do away with direct invites entirely and make all slots into qualifier slots. But I think there are certain merits in having direct invites, especially if they are based on known criteria, and ultimately this is a debate that is worth an entire article just for itself.
Which third party events?
This is mostly an issue of clarity. We’ve been told there are certain necessary requirements for an event to qualify as being a Minor or a Major. These requirements are based on prize pool and qualifiers. But are those requirements sufficient or there more considerations which have not yet been disclosed? Will Valve continue to exercise some discretion when vetting which events can win bids? It actually seems important that they do in an effort to discourage shady events from continuing to rip off teams and sabotage the game.
So there are good reasons to have more requirements than just prize pool and qualifiers. However, if those requirements are not disclosed, yet another transparency issue arises, which in turn becomes a problem of legitimacy or fairness. Just like we don’t want a barrier to entry for players, we also don’t want a barrier to entry for new events. Event organisers should be awarded bids based on equal grounds to one another. As with the first issue raised, this is a problem both because it opens the possibility of abuse and collusion and at a general level because it ultimately skews the qualification system to certain events or regions (this, in turn, will actually reinforce the problem of events having to discretion to choose direct invites).
Solution: The solution to this is simply to be open about all the criteria for winning bids. Even if one of the criteria is a process of vetting that is not completely transparent, just knowing that it’s part of the process is an improvement because independent parties can always make their own assessments as an accountability mechanism.
Money literally buys power
Qualifying Points will be awarded based on the total prize pool of a tournament, and whether the tournament is a Major or a Minor, with Majors giving more points per prize pool dollar.
This is a quote from Valve’s announcement. While scaling points based on prize pool is a good idea, there should probably be certain restrictions on this process. Obviously it’s great to incentivise competition and it’s likely to ultimately encourage more and more investment in the game. But I also think it might be wise to avoid a situation where event organisers can literally buy influence in the scene. This is an even more serious problem when coupled with the first problem raised in this article.
While not a likely outcome in the near future, a simple reductio illustrates this problem well. Imagine an event puts down so much money that the total points won at that event are worth more than all other events put together. Obviously that’s bad for the ecosystem because other events won’t be taken seriously — see the much discussed TI effect in China. While this kind of extreme might not arise anytime soon, it is not difficult to imagine problems arising based on drastically different prize pools.
Another way to think about this is in terms of our current conception of which events are the most important. I certainly think that a significant aspect of this is which teams are in attendance. Under the new system you could theoretically have a 16 team event with the mandatory 6 qualifier slots and 10 direct invites that all go to teams who aren’t in the top 16 rated teams in the world. If this event had more money than another event whose direct invites were directly based on the same rating system, it would seem bizarre to say that the event with more money but far weaker teams matters more than the event with less money but far stronger teams.
Solution: As suggested at the beginning of this section, the solution to this problem likely lies in setting caps to how many points events can be worth.
There is a second category of problems that the new system brings with it. This relates to the incentives it produces and the possibilities it enables with respect to roster formation.
Roster could become less stable
Because all that is required to qualify for TI is your top three players to have enough points, there is nothing to stop any three such players from forming an entirely new team shortly before the final roster lock and thus qualifying for the event. This is absurd because it means a team might qualify for TI having played zero competitive matches as a team. It’s also just a system that allows for severe roster instability. We’ve already seen how big teams like EG and Secret were willing to go through open qualifiers in order to make late roster changes so it’s quite plausible to believe that some players will want to do the same in the new system and this system actually doesn’t punish it at all, provided your players have earned enough points throughout the year. Dota players can be fickle at times, and teams can implode very quickly after a few bad results or internal issues. We certainly don’t want to encourage these kinds of possibilities if we’re interested in roster stability and the roster lock system seems to have been designed for that exact purpose.
Solution: There is a fairly simple solution to this problem. Require any team that qualifies for TI to have earned a certain percentage of their points while playing together. This could be done directly by setting a number or indirectly by placing the roster lock sufficiently early on that it’s not possible or extremely difficult to earn enough points before it (so at least if teams somehow did so they would be very unlikely to want to change their players). We have already been told that point values increase nearer to TI, so I’d say we’re halfway there. If the scaling fairly steep, it would probably solve this problem on its own.
Participation is no longer required
Although only considering the top three players is a stroke of genius in terms of enabling upward mobility, it also has the consequence of allowing for players who don’t participate in any professional Dota all year to simply walk into TI.
This is a dream come true for the Xiao8’s of the world. I think the whole point in establishing a fair and transparent qualification system is to guarantee that all those attending TI actually deserve to be there, that they earned their place at the event. And I think it’s patently obvious that a player who has been afk all year has not earned anything of the sort.
Solution: Again, there’s a fairly straightforward solution to this problem. In fact the above solution already solves it if teams are required to earn a minimum number of points together before qualifying for TI. An alternative way to solve this problem might be to simply set a minimum amount of participation required to qualify for TI, much like Magic: the Gathering’s ‘planeswalker points’. This rule would have to be fairly flexible in what it considered to be competitive events, as limiting it to only Minors and Majors would introduce an unreasonable barrier to entry.
I do want to reiterate that I’m really happy to see the changes Valve have made. Despite what people who read my articles may think, I don’t hate Valve. The reason I write these criticisms is because I work in this industry, I’m passionate about it, and I want it to be the best possible version of itself. It is my hope that the issues I have raised will be addressed in due course so that the progress Valve have made is not undermined by unintended consequences.