ProTest Labs
In Dota2 leagues and tournaments, there is a lot of emphasis on the draft before, during, and after the match, by analysts and commentators. During the draft phase, from the analyst desk, we hear a lot about how some hero pick is good or bad. During the game, certain picks will be praised, while others criticized, even categorized as crucial for the team’s win or defeat, and finally after the game, picks are sometimes the main focus of the conversation.
Knowing the complexity of a Dota2 game and all the possible elements which are responsible for a team winning the game, we decided to test how much drafts actually matter.
To do that, we are testing the relationship between background, skill, and draft preferences. We created a simple google questionnaire where we first asked people about their relationship with Dota.
After that, we showed 100 pairs of dota2 drafts. All of these drafts were actually real games from Dota2 DPC Upper Divisions, and all played on patch 7.29c. According to betting sites, there was no big favorite ( > 75%) in most of the games we picked. By doing that, we were left with games in which both teams were considered similar in skill.
We shared our questionnaire on Twitter, and with the (big) help of Noxville and FroggedTV, got 133 responses. If it took a minute to pick a favorable draft for each game, it would have taken 100 minutes to finish, which in this fast-paced world is a big chunk of time to spend on a dota questionnaire. We are extremely grateful to everyone who participated.
Results
After removing one 99/100 score (sorry m8, we think you are a *little bit* suspicious), the average of the correctly predicted drafts was 52.1. We were lucky enough to have 5 casters and 9 PROs doing our questionnaire. Casters averaged 49.2 correct answers, and PROs averaged 57.4 correct answers. Ranked players averaged 51.6 correct answers. Interestingly, 17 people who said they play Dota 2 but did not share their MMR averaged 54.8 correct answers, while 9 watchers averaged 53.6. Between 99 people who shared their MMR, we see a clear, gentle linear regression slope.
Overall best performing person in the whole questionnaire was Noxville, with impressive 69 correct answers. Good job! Does he know all the games that were played, or is he a Dota2 mastermind?
Fun facts
The most obvious draft combination to choose from was
where 115 of you voted correctly for draft no2. which was actually a SEA upper division Fnatic win over OB.Neon
The hardest one was
and only 30 of you guessed it correctly. The game corresponds to NA upper division game Sadboys’ win (draft 1) vs The Cut (draft 2).
Finally, the game where 66 of you answered for the first draft and 66 for the second one was
where SG e-sports (draft 2) won over Hokori (draft 1) in the SA upper division game.
Our verdict
This questionnaire is not detailed enough to conclude things, but it is enough to think about certain concepts we have about drafts in Dota 2. It is clear that PROs were the most accurate ones with 5.3 more correct answers than the group average and 5.7 more than non PROs, but is that enough? Overall average of 52.1 tells us that people(especially casters) cannot actually guess the correct draft, which doesn’t come as a big surprise, but why is that? To be fair, pro games are as balanced as it gets regarding drafts, but if even high-ranked MMR players and PROs don’t see a big difference between losing and a winning draft, how important are drafts really? We hear people using “outdrafted”, but should we read it as “this team just stomped them last game, so I am going to pretend that this draft is so much better”? Should teams try out crazier drafts and focus more on other aspects of the game like execution, decision making, prioritizing, etc.? What do you think?
Future plans
Our questionnaire was far from perfect, we decided to do it as a fun little activity before Major, and we had to hurry in fear of a new patch and people forgetting the actual state of heroes. Here are some problems and things that we will improve for next time:
- 100 drafts is way too much, people lose motivation quickly and give worse/random answers as time progresses and even give up. Average close to 50 could be attributed to that simple fact. We plan to change it and give you only ~10 random drafts from a pool of 100+ drafts, that way you can concentrate more on the drafts you got(and if you are well rested go for more) and we can hopefully get more answers.
- Sometimes players don’t favor any draft, and sometimes they are far more sure for some games than for others. Having just 1 or 2 options is not enough, so we plan to upgrade it by adding an “it is even” or make a simple percentage slider.
- Google questionnaire is not the most convenient method and will not be enough for these improvements, so we will make it a web app.
We hope you enjoyed our simple analysis of the relationship between background, skill, and draft preferences. If you have any ideas or comments, don’t hesitate to contact us and you can expect the next draft questionnaire right before TI!
