Deckstats Forum
English-language Forums => Commander Discussion => Topic started by: Morganator 2.0 on April 16, 2019, 02:39:49 am
-
Previously on Deckstats...
Looking at this from a statistical point of view: being short on spells is better than being short on lands (if you need to choose) because usually there are more spells than lands in your deck. Idk if you guys watch Command Zone but they actually did the math on this so it's not just my gut feeling about this: winning players tend to have most land in play.
I would like to see the stats that The Command Zone has.
They go over sample size etc at the beginning, but they don't exactly give all of the data they get. By this I mean if they ran an ANOVA, T-test Chi-Square (Absolutely no reason to run this one with this type of data) or similar, which I assume they did as they hired a statistician to analyze the data, they didn't give us the correlation coefficient, or a P value to describe if the results were statistically significant or insignificant. They just present the final data. For example: They state having white in your deck leads to your chance to win decreasing by 1% (assuming you start with a 25% chance to win in a 4 player game) and playing red increases it by 3%, and blue green and black were around 8% I think.
So here is the link to part 1: https://www.youtube.com/watch?v=Iwdb_kPCwNU
This is the second video: https://www.youtube.com/watch?v=ttGjuNXWxpY
Oh and they cover the price of the decks and their win% too.
Do you know what my favorite thing about Deckstats is? The stats.
Do you know what my most hated thing about The Command Zone is? Their stats.
First off, their data set is super incomplete. There are some instances where the number of lands was just left blank. This doesn't mean that there was no lands (I checked a couple of the videos), it just wasn't recorded. There were two games where mass land destruction was involved (I included those games). I also excluded games where there was no winner, because in all cases we are comparing who won.
But this is still an amazing data set to work with, and I applaud everyone who put this together. It's a big data set, so short of cEDH games, the sample is a good representation of the population.
Question 1: Does having more lands in a game cause you to win?
Null Hypothesis: There is no relation between the number of lands you play and if you won (-0.7>Correlation coefficient<0.7)
Alternate Hypothesis 1: Decks with more lands in play are more likely to win the game (Correlation coefficient>0.7)
Alternate Hypothesis 2: Decks with less lands in play are more likely to win the game (Correlation coefficient<-0.7)
There is an expression among statisticians; If you torture (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Torture) the data enough, you can make it talk. Which is why you want to avoid torturing data, lest you show that green jellybeans cause acne.
Believe me, I tortured this data for a long time. I could not get it to say that the players with more lands in play were more likely to win.
First I just ran the correlation of "Mana producing Lands at end of the Game" versus "Player Won?". So this is comparing across all games (n=304) if the player who won had the most lands. Correlation coefficient= 0.204, so there is no correlation between number of lands in play and who won. But then I did some things I wasn't supposed to (I tortured the data). I started by averaging the number of lands within games, to make a proxy for game length. So if a game had players with 15, 19, 14, and 16 lands, the average was 16, so the game was about 16 turns long. This is unlikely to be the actual game length (keep in mind I'm not supposed to be doing this), but it's a proxy. I then ran the correlation again, this time controlling for game length, to see if players ahead of the mana curve did better. Correlation coefficient= 0.275, so again, no correlation. Finally (really pushing it this time) I did within game correlation. So within each game, did the winning player have the most lands. Correlation coefficient= 0.218, once more no correlation!
Conclusion: I failed to reject the null hypothesis. I can say with confidence that there is no relation between the number of lands you play and if you win.
Interpretation: I think the problem with this analysis is that it only looked at lands. As I said before, mana sources would give a different result. Also, there are a lot of cEDH decks (namely Flash (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Flash) Hulk and Godo) that can easily win with only two lands, but with that early a win, everyone would have 2 lands.
Question 2: Does having Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) or Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) within your first 3 turns cause you to win more often?
Null Hypothesis: Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) and/or Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) in your first 3 turns does not have an effect on you winning (-0.7>Correlation coefficient<0.7).
Alternative 1: Players with Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) and/or Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) in their first 3 turns are more likely to win (Correlation coefficient>0.7).
Alternative 2: Players with Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) and/or Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) in their first 3 turns are less likely to win (Correlation coefficient<-0.7).
So I should get this out of the way; this null hypothesis sucks. I just can't think of a better way to phrase it. We know that Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) improves the power of your deck, that's why everyone uses it. So this is more measuring the strength of having this early game fast mana.
Running the simple (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Simple) correlation of "If there was a Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring)/Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt)" versus "Did that player win?" gives a correlation coefficient of -0.019, so no correlation. But because Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) is such a common card, I frequently saw games where 3 players all had Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring)/Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt), but only one person can win. So this time around, I think it's fair to transform the data. Next I compare "Did the player that won have a Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt)/Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring)?" and this is something for a Chi^2 test to handle. A Chi^2 test compares what was expected due to chance (the null hypothesis) compared to what actually happened. The math bit is a little complicated for me to explain, but if you're interested, this was the result.
| | Win | Loss | Total |
| Had a ring | Actual | 25 | 87 | 112 |
| Expected | 27.91 | 84.09 | |
| No Ring | Actual | 278 | 826 | 1104 |
| Expected | 275.09 | 828.91 | |
| Total | | 303 | 913 | 1216 |
So instead of me describing how I got to the p-value (0.505 by the way, so not significant). We can just look at the numbers. All the numbers we expect to get are very close to what we actually got.
Conclusion: I failed to reject the null hypothesis. There is no relation between you winning and if you played a Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) in the first 3 turns.
Interpretation: I think this question was asked the wrong way. What it actually should have been is "Do decks with Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) win more often then those without?" The issue is that budget would have an effect (most of the time people don't use Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) because they just don't have one).
Question 3: Which color is the strongest?
This is the point where I really get mad at the way this data set is organised. I'll be back in a few hours to finish this post off.
Tonight on Deckstats, I present the analysis to figure out which is the best color in Commander, based on the data from The Command Zone and I also compare which is the best color combination out of all 32.
You know, I just realized… The Command zone paid people to do these stats, and I’m doing it for free.
Step 1: I simplified all of the data so that it made sense.
Decks containing…
White= 537
Blue= 578
Black= 594
Red= 549
Green= 584
Colorless= 7
Number of each deck in each color identity
Colorless= 7
Mono-White= 41
Mono-Blue= 58
Mono-Black= 75
Mono-Red=80
Mono-Green=59
Azorius= 38
Dimir= 45
Rakdos= 42
Gruul= 42
Selesnya= 44
Orzhov= 34
Izzet= 29
Golgari= 46
Boros= 27
Simic= 48
Esper= 35
Grixis (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Grixis)= 41
Jund= 25
Naya (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Naya)= 31
Bant (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Bant)= 47
Abzan= 30
Jeskai= 29
Sultai= 36
Mardu= 37
Temur= 27
Anti-Green= 14
Anti-White= 19
Anti-Blue= 18
Anti-Black= 15
Anti-Red= 24
5-color= 73
Part 1: Which color is the best.
I hate this question. If we’re defining the best by number of wins per game played, then the order goes Blue, Green, Black, Red, White. But, we don’t know if this is a significant difference or not, these numbers are really close to each other. For that we use the Chi squared test again. I’m going to skip over most of the math bits this time (but I will show them if someone asks). The p-value of the chi-squared test was 0.162 which is not significant. Therefor with the data presented, we cannot say which is the best color. And I think this comes down to how I had to do this. Because decks can be more than 1 color, things get wonky. So instead, lets look at color identity.
Part 2: Which color identity is best.
This is literally the same thing as before, just with 32 levels instead of 5. But same deal, the p-value of the chi-squared test was 0.216, so not significant.
In case you're interested, I've attached the graphs that show the number of wins per game. Keep in mind that because these results were non-significant, if we played another 300 games of commander, we would see different results.
I know this doesn't look like much, but this took hours (mostly just rearranging the raw data so it makes sense). I'll leave interpretation for later, because we kinda know from experience that white is the worst color.
-
Notice the order based on the data is exactly what I said. The differences may not be significant, but they reinforce the order that we all know, which I think gives it credibility. Also, most of the better performing color combos had black, blue, green, or some combination of them.. I think red and white are close to each other and the sultai colors are close to each other but red and black are pretty far apart, from experience.
Also..
SANS-Green= 14
SANS-White= 19
SANS-Blue= 18
SANS-Black= 15
SANS-Red= 24
:P
btw r u like some kind of statistician lmao
-
btw r u like some kind of statistician lmao
Amateur statistician I guess. My field of study requires me to know about statistics, and I translate my knowledge of statistics to card games.
In other words, I'm not sexually active.
But I'm not done yet. There is still one other thing that's been bothering me.
For example: They state having white in your deck leads to your chance to win decreasing by 1% (assuming you start with a 25% chance to win in a 4 player game) and playing red increases it by 3%, and blue green and black were around 8% I think.
I'm still not quite sure how they came to this conclusion. Once I get some proper sleep (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sleep) I'll lock myself in a dark room to figure this out.
-
Love this thread and the series of posts in the other thread that lead to it!
Nothing surprising here as far as wide-spread perception of the best/worst colors. I'm curious to see what happens when you breakdown the CZ's numbers of increased/decreased win probability based on color inclusion.
-
So I did something new now; I watched the first video.
At least, the first 20 minutes. These videos are extremely boring. Are there actually people who like this?
But anyway, I found out that the person they hired does have experience with Magic, but not with commander. Honestly.. good enough.
Here is the issue though. The two hosts of this video did not present the stats correctly. At all. I just know that this is going to come back to bite me in the future. At some point, I am going to spend a long time explaining to someone at my local game store that they shouldn't make a Planeswalker deck just because of The Command Zone's stats.
But I digress. What really caught my attention was that the numbers were extracted from the data set with a Python (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Python) script. That worries me. For something like this you really should use a proper statistical software like R Studio (my preference), SPSS, Mini-Tab, or even Microsoft Excel (for the simple (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Simple) analyses). Still, this guy has a Harvard education, so he should know what he's doing (I now also understand why he got paid).
But the other thing that caught my attention; none of these graphs have error bars. So while a bar graph shows where the data did land, the error bars show where the data could have landed, which is important for statistical significance. As a general rule, if the error bars cross, the data is non-significant. Here's an example:
(https://i.imgur.com/LYimLow.jpg)
Continuing with the trend of people wondering what I do on my spare time, this is a graph I made showing the height of Golden Rod flowers, where some have been parasitized (the ones with galls). You can sort of make out that the plants that have been parasitized are slightly shorter, but because the error bars cross, we can't conclude anything. If I had done these measurements again, with the same number of plants, then I could see that the parasitized plants were slightly taller.
Now this might seem hypocritical, because the last two graphs I posted to this thread didn't have error bars. That's because it was late for me, and while Excel can put error bars on graphs, it is not good at it... like... at all. You really have to smack it around to make it work. Instead, I ran the Chi squared test for significance.
Okay, enough ranting (for now). Here are the graphs that The Command Zone showed.
(https://i.imgur.com/C18lZIC.jpg)
So while I haven't checked, I have a hard time believing that all the people without Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) landed exactly on 25%. That seems like a fudged number.
(https://i.imgur.com/n36udFF.jpg)
(https://i.imgur.com/ySCpYZi.jpg)
(https://i.imgur.com/0Ugukiz.jpg)
(https://i.imgur.com/XPMot0z.jpg)
So this picture isn't actually a statistic, it's just showing how each deck was defined in terms of play style. So you are an enchantment deck if you have 20 or more enchantments.
(https://i.imgur.com/m8Rbf63.jpg)
The more I see of these graphs, the less convinced I am that these are well-tuned decks. No way does combat damage do better than combo.
(https://i.imgur.com/QWPYqEX.jpg)
Now this might be me being nit-picky, but I'm pretty sure the numbers on this graph are wrong. 18%*3+42% makes a total of 96%. You can't just have 4% go missing.
Without p-values or correlation coefficients, these numbers mean nothing. But I'm going to leave these graphs here. I'm going to see if I can re-create them in the future, with error bars.
-
Speaking of stats, the probability calculator on here sucks. For a 1 of in EDH it goes 7/100 for the probability in the opening hand, and adds 1/100 for each subsequent turn. Not only are there only 99 cards in the actual deck, but the denominator decreases by 1 each time. For example, It says a 7% chance of drawing a card in the opening hand. Well I did the math and it's actually 7.3% It also says that there's a 17% chance for turn 10. Well I did the math again ((1/99)+(1/98)+(1/97)+(1/96)+(1/95)+(1/94)+(1/93)+(1/92)+(1/91)+(1/90)+(1/89)+(1/88)+(1/87)+(1/86)+(1/85)+(1/84)+(1/83)) and it's actually 17.5%
p.s. I hope Nils reads this and gets a real hypergeometric calculator put into deckstats.
p.p.s. this is obviously nothing compared to Morganator's math but I found this very annoying
p.p.p.s. correct me if this is completely wrong but the numbers are close so I'm gonna assume it is right. I hope I haven't forgotten basic math bc it hasn't been THAT long.
-
Speaking of stats, the probability calculator on here sucks. For a 1 of in EDH it goes 7/100 for the probability in the opening hand, and adds 1/100 for each subsequent turn. Not only are there only 99 cards in the actual deck, but the denominator decreases by 1 each time. For example, It says a 7% chance of drawing a card in the opening hand. Well I did the math and it's actually 7.3% It also says that there's a 17% chance for turn 10. Well I did the math again ((1/99)+(1/98)+(1/97)+(1/96)+(1/95)+(1/94)+(1/93)+(1/92)+(1/91)+(1/90)+(1/89)+(1/88)+(1/87)+(1/86)+(1/85)+(1/84)+(1/83)) and it's actually 17.5%
p.s. I hope Nils reads this and gets a real hypergeometric calculator put into deckstats.
I noticed this a while ago, but I didn't think that there were this many people that cared about statistics. The hypergeometric calculator is fine, but with commander decks, it actually counts the commander(s) as being cards you could possibly draw. I've wondered if this is why so many people put their commanders in the side board.
-
It's not even just the out of 100. Even with that, it should add (1/10)+(1/99) etc for the statistics but it just multiplies 1/100 by however many cards have been drawn, which is NOT how it works.
Side note, I do add the probabilities together right.. lol I despise (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Despise) stats
-
I think I just watched "A beautiful mind" on paper ;D ;D , but 1 quick question , I might have missed this in the previous posts , is this all based off 1 vs 1 or 3 player , 4 player ect and how much would that alter all this information the more or less (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=More+or+Less) people playing when the data was recorded ?
-
The more I see of these graphs, the less convinced I am that these are well-tuned decks. No way does combat damage do better than combo.
Yeah I agree with you here Morgantor. From my interactions with you guys about the more competitive side of EDH, I have noticed that all these videos of EDH games are casual players with casual level decks. The only reason they might be better than other casual player decks is because they have all 10 original duals, and they have force of will (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Force+of+Will), etc.
p.p.s. this is obviously nothing compared to Morganator's math but I found this very annoying
Haha. You said P P.
Side note, I do add the probabilities together right
It is funny how numbers work sometimes. Like a coin coming up heads/tails is 50/50, but if you get heads the first flip, it doesn't make the odds (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Odds) of the next flip being tails 100%. It is still 50/50 because the previous flip(s) don't affect the last one(s), but when drawing a deck, each draw does affect the probably of the next card being. For example, we have a two card deck with an ace of spades and ace of hearts. The probability that the first card drawn is an ace of spades is 50/50. If we draw the first card, regardless of what it is, we know what the next card will be (or rather won't be, and since it is a two card deck, we can use deductive reasoning because we is smart.)
is this all based off 1 vs 1 or 3 player , 4 player
All the data from the command zone episode, which is what this thread is about, (See, staying on topic Morgantor, proud of me?) was from a 4 player game if I am not mistaken.
Okay, so I couldn't figure out how to quote the image, without getting the entire post, so I cheated and took a screenshot.
So what is non combat no combo? Is that killing with an ability like purphoros, god of the forge (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Purphoros%2C+God+of+the+Forge), with his ETB effect?
-
Side note, I do add the probabilities together right
It is funny how numbers work sometimes. Like a coin coming up heads/tails is 50/50, but if you get heads the first flip, it doesn't make the odds (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Odds) of the next flip being tails 100%. It is still 50/50 because the previous flip(s) don't affect the last one(s), but when drawing a deck, each draw does affect the probably of the next card being. For example, we have a two card deck with an ace of spades and ace of hearts. The probability that the first card drawn is an ace of spades is 50/50. If we draw the first card, regardless of what it is, we know what the next card will be (or rather won't be, and since it is a two card deck, we can use deductive reasoning because we is smart.)
It's the total probability of drawing it BY a certain turn. For the probability of drawing it ON a certain turn you don't add them, but u would add them to see the total probability of having drawn it by that turn.
-
Non-combat non-combo is exactly like Purphoros (that was even the example that was used). It could also be something like Gray Merchant of Asphodel (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Gray+Merchant+of+Asphodel).
And the "Other" category is a card like Felidar Sovereign (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Felidar+Sovereign) or Helix Pinnacle (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Helix+Pinnacle). It's worrisome that those were the examples used instead of Laboratory Maniac (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Laboratory+Maniac) and Approach of the Second Sun (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Approach+of+the+Second+Sun).
-
I mean I have no doubt they're completely casual decks..
-
Non-combat non-combo is exactly like Purphoros (that was even the example that was used). It could also be something like Gray Merchant of Asphodel (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Gray+Merchant+of+Asphodel).
And the "Other" category is a card like Felidar Sovereign (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Felidar+Sovereign) or Helix Pinnacle (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Helix+Pinnacle). It's worrisome that those were the examples used instead of Laboratory Maniac (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Laboratory+Maniac) and Approach of the Second Sun (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Approach+of+the+Second+Sun).
Isnt laboratory maniac (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Laboratory+Maniac) a combo win condition? Or I guess by combo you guys mean specifically infinite combos instead of like some doomsday (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Doomsday) pile combo.
-
(https://media1.tenor.com/images/1919516d53b660872005debdebc8c391/tenor.gif?itemid=10311874)
I can't believe this.
I can't believe how long it took for me to figure this out.
You want to see what overthinking something looks like, scroll up to the top of this page and re-read all of it. I completely overthought all of this. Had I just remembered my first day of stats class, none of this would have happened. This is literally the first thing you learn.
Let me explain...
The first thing you learn about in statistics is the difference between a sample, and the population. The population is everything. If you are looking at the average height of people worldwide, the population is everyone. Every person on Earth. If you are looking at the average height of people in the United States, then the population is everyone living in the United States. And if you are examining the win percentages of commander games, the population is every possible game you could ever have. Every commander, every deck variation that commander could have, every possible combination of decks that could go against each other, and every possible outcome of those games.
As you can imagine, it is impossible (or at least hugely impractical) to measure the population.
So instead, you take a sample. The sample is where you only measure a group people, or you only look at a some commander games, not all of them. But with a good enough sample, you can accurately estimate the properties of the population. Now I thought that with 304 commander games, the Command Zone had a good sample size. And they do; 300 or so games is a very good sample. I've said this already.
Where I messed up, is that I thought that the sample was estimating all commander games in existence. But I watched a few of the games, I saw the decks that were being used. And then it hit me:
These are YouTube videos, they are being done for entertainment. That's why combat damage was the most common win condition; watching someone win with a combo is boring, but with combat damage, it's more entertaining to see someone pull ahead. That's why Helix Pinnacle (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Helix+Pinnacle) is a more common win condition than Approach of the Second Sun (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Approach+of+the+Second+Sun); it's more entertaining to watch. Those 304 games aren't a sample of all possible commander games, they are only a sample of the possible games made by YouTubers.
I can't believe I didn't think of this sooner.
(https://images.gr-assets.com/hostedimages/1380427711ra/855157.gif)
Summary
Wrapping this up now, I'm not going to spend any more time on it.
- All results for The Command Zone's stats were found to be insignificant. Just because the correlation coefficient was -0.19, doesn't mean that Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) decreases your win chance.
- None of the results found by The Command Zone will reflect your games. I mean, they might, but it's not likely.
Right, I'm going to lay of the stats for a bit now. I'll wait until something else catches my eye.
So, give it an hour.
-
(https://media1.tenor.com/images/1919516d53b660872005debdebc8c391/tenor.gif?itemid=10311874)
I can't believe this.
I can't believe how long it took for me to figure this out.
You want to see what overthinking something looks like, scroll up to the top of this page and re-read all of it. I completely overthought all of this. Had I just remembered my first day of stats class, none of this would have happened. This is literally the first thing you learn.
Let me explain...
The first thing you learn about in statistics is the difference between a sample, and the population. The population is everything. If you are looking at the average height of people worldwide, the population is everyone. Every person on Earth. If you are looking at the average height of people in the United States, then the population is everyone living in the United States. And if you are examining the win percentages of commander games, the population is every possible game you could ever have. Every commander, every deck variation that commander could have, every possible combination of decks that could go against each other, and every possible outcome of those games.
As you can imagine, it is impossible (or at least hugely impractical) to measure the population.
So instead, you take a sample. The sample is where you only measure a group people, or you only look at a some commander games, not all of them. But with a good enough sample, you can accurately estimate the properties of the population. Now I thought that with 304 commander games, the Command Zone had a good sample size. And they do; 300 or so games is a very good sample. I've said this already.
Where I messed up, is that I thought that the sample was estimating all commander games in existence. But I watched a few of the games, I saw the decks that were being used. And then it hit me:
These are YouTube videos, they are being done for entertainment. That's why combat damage was the most common win condition; watching someone win with a combo is boring, but with combat damage, it's more entertaining to see someone pull ahead. That's why Helix Pinnacle (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Helix+Pinnacle) is a more common win condition than Approach of the Second Sun (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Approach+of+the+Second+Sun); it's more entertaining to watch. Those 304 games aren't a sample of all possible commander games, they are only a sample of the possible games made by YouTubers.
I can't believe I didn't think of this sooner.
(https://images.gr-assets.com/hostedimages/1380427711ra/855157.gif)
Summary
Wrapping this up now, I'm not going to spend any more time on it.
- All results for The Command Zone's stats were found to be insignificant. Just because the correlation coefficient was -0.19, doesn't mean that Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) decreases your win chance.
- None of the results found by The Command Zone will reflect your games. I mean, they might, but it's not likely.
Right, I'm going to lay of the stats for a bit now. I'll wait until something else catches my eye.
So, give it an hour.
So can we conclude that amount of available mana affects win %, or price of the deck? Just not to the degree calculated by the command zone?
Perhaps conclude is a bad word, but I think looking at the stats and common knowledge tells us this could be the case? We just cant assign a % value to them.
-
It's really not the case though.. Flash (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Flash) only costs 2 mana ::)
The best wincons are always cheaper
-
It's really not the case though.. Flash (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Flash) only costs 2 mana ::)
The best wincons are always cheaper
From my understanding this is the case in cEDH, but in a more casual setting, the game is usually won by 8+ mana spells. Expropriate (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Expropriate), cyclonic rift (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Cyclonic+Rift), Insurrection (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Insurrection), Craterhoof Behemoth (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Craterhoof+Behemoth). Big bombs like that. Not so much getting a combo off quickly.
-
cEDH is also the only place you'll get any sort of real meta or data on a scale larger than a single playgroup for EDH. All casual data will be skewed by a million different factors so it means next to nothing anyways
-
cEDH is also the only place you'll get any sort of real meta or data on a scale larger than a single playgroup for EDH. All casual data will be skewed by a million different factors so it means next to nothing anyways
That is a great point. Thinking outside cEDH, just casual, I think across most playgroups it's a general trend that more mana plays a factor in winning. Although if you've hit every land drop, but drew lands/ramp all game, I guess it doesnt matter your mana count if you cant use it.
-
cEDH is also the only place you'll get any sort of real meta or data on a scale larger than a single playgroup for EDH. All casual data will be skewed by a million different factors so it means next to nothing anyways
Be careful, data skewing happens in cedh as well.
-
Decks aren't skewed by budget or janky stuff the way EDH is. Everything is optimal and there's actually a meta.. much better for gathering data
-
So can we conclude that amount of available mana affects win %, or price of the deck? Just not to the degree calculated by the command zone?
Perhaps conclude is a bad word, but I think looking at the stats and common knowledge tells us this could be the case? We just cant assign a % value to them.
No. No to all of this.
You'll notice in the earlier posts I mentioned a null hypothesis. The null hypothesis is always nothing. "There is no correlation" or "there is no trend". The goal of statistics is to provide evidence that rejects the null hypothesis, so that you can give evidence to the alternate hypothesis; "there is a correlation" or "there is a trend".
In none of the tests I did was I able to reject the null hypothesis, so we must accept it; there are no correlations between anything.
And this comes down to the wide range of decks that were used; if you try to make a rule (ex: more lands increases your chance of winning) there are just too many decks out there that breaks those rules. You might find success if you narrowed the hypothesis that you're testing (ex: Boros decks with more lands have an increased chance of winning).
-
First we have to come to a conclusion as to what exactly constitutes as budget. Is budget $25.00? $50.00? $100 and so forth all the way into the thousands? The stats on command zone concluded as their opinion as budget below $500.
What is defined as budget can mean differently from person to person.
Second we also have to come to a conclusion what exactly constitutes as optimization. This also ranges based apon what one defines it at. We could call it best optimization at winning. Then that brings in stats command zone talks about but is also inconclusive. Does the chance of winning actually increase by going first? Does the chance of winning increase by who does play the most lands? Does the chance of winning increase by who gets the most mana? Does the chance of winning increase by what commander I choose? We can agree that some just won't no matter what because they just don't do much of anything. Does the chance of winning increase based apon color choosing? Established to an extent with the colors but nothing really conclusive. Does the chance of winning increase with how much a card cost? This is also part of the budget aspect. This also follows into the "keeping up with the joneses". Per wonderful example of Command Zone with Smothering Tithe (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Smothering+Tithe). People so when Josh had that on the field and he played wheel of fortune (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Wheel+of+Fortune). People jumped on the bandwagon/keeping up with the joneses on Smothering Tithe (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Smothering+Tithe). Does winning increase with skill? No brainer here but the more skill you have at something the better you are at it then someone who has just started. Even though there are exceptions here as well because some people can just get that skill really fast after having just started. Hence slow learners and fast learners.
Optimization has also been proven while disproven to not work. I hate to use this as an example but go into sports. Look at teams who have payed the most for optimization compared to those whom haven't. Yet some teams who haven't payed for optimization have gone (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Gone) out to win the big brew ha ha and so has those whom have paid.
When looking at things like these stats like those of the Command zone it gives some sort of general idea. Things are plausible but not conclusive. Until there are full complete studies on every single aspect of commander we will only have things that just give a general idea but nothing fully conclusive.
-
It's also hard to take some of this info seriously as some of the video series have soft ban lists. Cmdr Vs tries not to use Cyclonic Rift (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Cyclonic+Rift) or combo decks. Command Zone tries to stay away from early game winning combos. Both have themed games where some of the results wouldn't happen in your normal games. There's another one (I can't think of the name}, had a custom rule where you couldn't do an infinite combo more than so many times.
These video series play a 'fun to watch but not comparable to real games unless you house rule their exact rules' format.
-
But there is also another thing that wasn't mentioned. If I remember correctly from older videos Josh doesn't use tutors anymore. So how many of his games that are part of the stats are decks that he doesn't have tutors in.
Also are the decks the Command Zone also used based with the assumption of how they recommend how you make an EDH deck?
-
I'm not sure but I feel like I remember seeing Josh use a tutor in a recent Game Knights (and I'm not counting the most recent episode where his Boros deck was built around Sunforger (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sunforger)). I know Jimmy has.
ETA- maybe it was an older episode that I just re-watched for some reason?
-
I'm not sure but I feel like I remember seeing Josh use a tutor in a recent Game Knights (and I'm not counting the most recent episode where his Boros deck was built around Sunforger (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sunforger)). I know Jimmy has.
ETA- maybe it was an older episode that I just re-watched for some reason?
No no I remember him using a worldly tutor (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Worldly+Tutor) I think on an extra turns episodeand that's a fairly new series.
-
The latest Extra Turns was almost 5 months ago and not what I was thinking of.
-
The latest Extra Turns was almost 5 months ago and not what I was thinking of.
Well I dont remember him.saying at any point in the past 5 months he doesnt use tutors. So either way I think it just helps prove your point. I know in the game knights I think with their special guest dude Josh used defense of the heart (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Defense+of+the+Heart).
-
You're right. I was just saying that particular episode wasn't what I was thinking of. I could swear I remember him using a Demonic Tutor (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Demonic+Tutor) recently.
-
LOL I said if I remember correctly but however he did say something about tutors. The exact quote of what Josh says I am not fully sure of
-
LOL I said if I remember correctly but however he did say something about tutors. The exact quote of what Josh says I am not fully sure of
It sounds like something he would say, to not use tutors, but perhaps he was giving advice on how to power down your deck because some people have the problem of their decks being too powerful for their meta. So one way to stop that (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Stop+That) would be less tutors to make the deck more random.
-
Now to talk about my favorite thing Deckstats has to offer: the stats!
I would like to see the stats that The Command Zone has (I refuse to dig through YouTube videos looking for it). What was the value of the correlation coefficient (>0.70 is a fairly good correlation), what was their sample size, what were their error margins, and how did they measure which decks did better?
Ok, so this link leads to the analyst's own page: https://public.tableau.com/profile/andrew.greene#!/vizhome/CommanderStatistics/Title
Andrew Greene is a statistician the Command Zone hired for this purpose. They manually sifted through 300 commander games and looked at various variables and correlations in the data. The only way to get all the conclusions is to actually watch the videos (sorry), but the raw data dump is linked on Andrew's page. Here's the link: https://docs.google.com/spreadsheets/d/10c7mflt6FJ253rtKeFAbQhPT282JDzJ6BcwrOV5MIzo/edit#gid=0
Just by looking at the data dump you can see they tracked a lot of things. Their sample is obviously a little skewed because all the games as far as I understood were recorded games, which means they were played by a small group of people. This is not a cumulative archive of all games played since the beginning of time but the good thing is that we can go back and verify every data point from the recordings.
The way they did the conclusions were mostly in terms of win percentages. This means that in four-way games their base line was always 25% and they looked at things like land count in the deck, land count at the end of the game, T1 Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring), colors, deck cost and stuff like that. Then the charts would display an increase of decrease in percentage points compared to the 25% base line.
TL;DR n=316
I'm moving the conversation back here.
First off, n actually equals 304. Some of the games had incomplete data, and some of the games didn't have a winner. I excluded those games because if we're looking at win percentages, then the games should have a clear winner.
Second, I did download the data set and analysed it myself (next time I'm getting paid). The issue wasn't that the data was shown incorrectly; all of the graphs were accurate, the issue is that they didn't present the stats. In the parts of the video that I did watch, they would say something like "You can see the correlation; an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) makes you less likely to win." This isn't true though. I actually did the correlation and it wasn't strong enough to conclude that an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) hurts your chances of winning (correlation coefficient= -0.019).
So while they presented the data, they didn't present the statistics.
We cannot reject any of the null hypotheses, so we must accept them.
-
I'm moving the conversation back here.
Uhh... I have no idea what happened. I just went through threads that had "new" replies (i.e. ones I hadn't seen yet). I'm pretty sure I've never seen this thread and yet I'm somehow quoted in the first message.
TL;DR: Seems like you've addressed everything already and now I just look dumb for not noticing there's a thread I'm apparently a participant in?
-
Now to talk about my favorite thing Deckstats has to offer: the stats!
I would like to see the stats that The Command Zone has (I refuse to dig through YouTube videos looking for it). What was the value of the correlation coefficient (>0.70 is a fairly good correlation), what was their sample size, what were their error margins, and how did they measure which decks did better?
Ok, so this link leads to the analyst's own page: https://public.tableau.com/profile/andrew.greene#!/vizhome/CommanderStatistics/Title
Andrew Greene is a statistician the Command Zone hired for this purpose. They manually sifted through 300 commander games and looked at various variables and correlations in the data. The only way to get all the conclusions is to actually watch the videos (sorry), but the raw data dump is linked on Andrew's page. Here's the link: https://docs.google.com/spreadsheets/d/10c7mflt6FJ253rtKeFAbQhPT282JDzJ6BcwrOV5MIzo/edit#gid=0
Just by looking at the data dump you can see they tracked a lot of things. Their sample is obviously a little skewed because all the games as far as I understood were recorded games, which means they were played by a small group of people. This is not a cumulative archive of all games played since the beginning of time but the good thing is that we can go back and verify every data point from the recordings.
The way they did the conclusions were mostly in terms of win percentages. This means that in four-way games their base line was always 25% and they looked at things like land count in the deck, land count at the end of the game, T1 Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring), colors, deck cost and stuff like that. Then the charts would display an increase of decrease in percentage points compared to the 25% base line.
TL;DR n=316
I'm moving the conversation back here.
First off, n actually equals 304. Some of the games had incomplete data, and some of the games didn't have a winner. I excluded those games because if we're looking at win percentages, then the games should have a clear winner.
Second, I did download the data set and analysed it myself (next time I'm getting paid). The issue wasn't that the data was shown incorrectly; all of the graphs were accurate, the issue is that they didn't present the stats. In the parts of the video that I did watch, they would say something like "You can see the correlation; an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) makes you less likely to win." This isn't true though. I actually did the correlation and it wasn't strong enough to conclude that an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) hurts your chances of winning (correlation coefficient= -0.019).
So while they presented the data, they didn't present the statistics.
We cannot reject any of the null hypotheses, so we must accept them.
Forgive me. One quote says n=316 and you said its 304. I didnt go look for myself. What happened to those other 12 games? You said there was no clear winner? But why would you exclude (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Exclude) them? If I play 1 game with a deck and nobody wins, I lose, right? So my win percentage is 0% with that deck surely this is the case for those 12 games with no winner?
-
I don't actually know why those games didn't have a clear winner. If I had to guess it would be because some kind of time limit was being enforced. I really didn't want to re-watch those videos if it wasn't necessary. I've been to finance meetings that are more entertaining than those videos.
The reason why I excluded them is because we're looking at win rates and which factors cause decks to win. When this is the case, we want there to be a clear winner. Also, it messes with the Chi-squared tests. You see normally, Chi-squared assumes equal variance of outcomes: the null is that you are just as likely to flip heads as you are tails. The chi-squared tests I used assumed that there were two outcomes: win, which is one person, or lose, which is three. Not 50-50. If there is a possibility of draw (all three losing), which adds an extra factor to the Chi-squared distribution, which changes the null that we're testing. If we wanted to test the odds of a deck winning/losing/everyone losing, I would include them. But getting rid of 12 samples, while still keeping 304 good ones, means that the sample size is still strong.
What I should have maybe done is actually remove some more for the color analysis. There were 7 colorless decks, with 3 wins. That's not a good enough sample size to justify that colorless is one of the most powerful color combinations.
-
I don't actually know why those games didn't have a clear winner. If I had to guess it would be because some kind of time limit was being enforced. I really didn't want to re-watch those videos if it wasn't necessary. I've been to finance meetings that are more entertaining than those videos.
The reason why I excluded them is because we're looking at win rates and which factors cause decks to win. When this is the case, we want there to be a clear winner. Also, it messes with the Chi-squared tests. You see normally, Chi-squared assumes equal variance of outcomes: the null is that you are just as likely to flip heads as you are tails. The chi-squared tests I used assumed that there were two outcomes: win, which is one person, or lose, which is three. Not 50-50. If there is a possibility of draw (all three losing), which adds an extra factor to the Chi-squared distribution, which changes the null that we're testing. If we wanted to test the odds of a deck winning/losing/everyone losing, I would include them. But getting rid of 12 samples, while still keeping 304 good ones, means that the sample size is still strong.
What I should have maybe done is actually remove some more for the color analysis. There were 7 colorless decks, with 3 wins. That's not a good enough sample size to justify that colorless is one of the most powerful color combinations.
Colorless has a 43% win %. It is clearly the most powerful.
One other question (which may foster (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Foster) into several more, we'll see.)
How do you account for the same color being played by multiple people? I mean if we have 1 five color deck, one Boros, one Jund and one Grixis, red is guaranteed to win that match, but we all know having red in your deck d oesn't guarantee you will win.
-
I was worried someone might ask this. A little while back I figured out how they might have determined that white decreases your win chance by 1%, red increases it by 3%, and so on.
Question for those that watched the video. At any point, did someone say "Multiple linear regression"?
A multiple linear regression is an equation with multiple variables, that gives a weight to each variable. As a real-world example, this is one of the ways to determine wage gaps. You make "yearly gross earnings" the variable that the equation solves for, and then you can add the variables that would affect your pay. Some of these variables could be "time spent with the company", "level of education", "position in job", and so on. The cool thing about a multiple linear regression, is that you can tack on anything else you think might affect your pay. "Race", "gender", "sexual orientation". Each variable is then given a weight (how much it affects the pay), an error, and a significance value. So if "race" has a high weight, and has significance, that means that your employer does descriminate between race and pays employees differently because of it.
If we were to apply this to our stats, we would make "win chance" the dependent variable that we're solving for, and add each color as one of the variables. The equation might look something like this:
Win chance= 0.25+0.08xu+0.08xb+0.08xg+0.03xr-0.01xw+error
Each instance of "x (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=X)" represents one of the colors (denoted by the subscipt), and it would either be 1 (if the deck has the color) or 0 (if it doesn't).
So I could do this... but I kinda don't want to. As you can imagine, this would take a lot of work.
What I want to know first is if The Command Zone did this. I only ever saw that bar graph, which is not a good substitute for a multiple linear regression. Did the stats guy that they hired do anything like this?
-
I was worried someone might ask this. A little while back I figured out how they might have determined that white decreases your win chance by 1%, red increases it by 3%, and so on.
Question for those that watched the video. At any point, did someone say "Multiple linear regression"?
A multiple linear regression is an equation with multiple variables, that gives a weight to each variable. As a real-world example, this is one of the ways to determine wage gaps. You make "yearly gross earnings" the variable that the equation solves for, and then you can add the variables that would affect your pay. Some of these variables could be "time spent with the company", "level of education", "position in job", and so on. The cool thing about a multiple linear regression, is that you can tack on anything else you think might affect your pay. "Race", "gender", "sexual orientation". Each variable is then given a weight (how much it affects the pay), an error, and a significance value. So if "race" has a high weight, and has significance, that means that your employer does descriminate between race and pays employees differently because of it.
If we were to apply this to our stats, we would make "win chance" the dependent variable that we're solving for, and add each color as one of the variables. The equation might look something like this:
Win chance= 0.25+0.08xu+0.08xb+0.08xg+0.03xr-0.01xw+error
Each instance of "x (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=X)" represents one of the colors (denoted by the subscipt), and it would either be 1 (if the deck has the color) or 0 (if it doesn't).
So I could do this... but I kinda don't want to. As you can imagine, this would take a lot of work.
What I want to know first is if The Command Zone did this. I only ever saw that bar graph, which is not a good substitute for a multiple linear regression. Did the stats guy that they hired do anything like this?
I don't quite remember. I watched the videos upon release (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Release) which was what? October or something. Most of the videos were their theorizing why the data looks like this. For instance, they concluded that sol ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) in the first 3 turns decreases your chances to win (We know this isn't accurate now), but then they spent like 8 minutes talking about how playgroups may be self correcting for the early advantage of sol ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) by attacking that player. They would give examples like this, "Hey, you have a soul ring, I'll attack you with my little 1/1. Any of the data analysis they talk about is probably done in the first 10 minutes of the first video as they explain everything, and I don't remember hearing multiple linear regression. I doubt they mentioned it because the majority of their audience has no clue what it means, while some have only a faint idea, and there is the .5% that are statisticians like apparently you, Morganator .
-
This isn't really scientific but I've watched a few of the games and I have a gut feeling like some of the players play very differently compared to many play groups. They focus on politics as if they were role playing: hold grudges and punish people. I haven't experienced this in our play group (insignificant sample size :P) but I can't also imagine that to be unique. Command Zone speculated on the idea that having a Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) early would make them a generic "threat" that people respond to. This isn't true unless, of course, you play on YouTube and your ad revenue depends on the amount of drama and tension you can generate. Thus it would only make sense that they're role playing their games.
Josh sometimes does point out that some of his moves were just 'the best play' but the Professor for example often makes decision based on his idea of justice (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Justice) and his gut feeling. The Professor has stated multiple times that his objective is not to win, but to play an interesting game. There could be an argument made here that his games should be excluded from the sample how is something going to affect win rate is an insane question to ask if some people aren't even necessarily trying to win in the sense that they're willing to sacrifice the best possible play for more interesting and engaging play.
***
I thought of something we haven't discussed yet: ratios of colours in a deck.
Instead of looking at decks containing colours X (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=X), Y or Z we could look at the cards' colour identity. I don't know if it would be best to exclude (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Exclude) lands or not.
Here's an extreme example: Samut, Voice of Dissent (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Samut%2C+Voice+of+Dissent) is a Gruul card with white tacked onto it. You could, in theory, play a Gruul deck with no way to produce white mana and never use the ability. This wouldn't be evident from the colour identity but it would be very evident from the individual cards being played; the deck would show up as a Naya deck even though it's closer to a Gruul deck.
This would also yield a larger sample size for colourless since every Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring), every Burnished Hart (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Burnished+Hart), every Scour from Existence (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Scour+from+Existence) and every Lightning Greaves (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Lightning+Greaves) would actually count towards the colourless sample. You could then compile the individual cards into pools of 6 colours or 32 colour combinations.
I think this would better reflect the power of each colour, in a commander deck.
Most of the matches do contain the link to the deck list so in theory this could be done. I'm not actually asking anyone to do it, just toying around with the idea.
-
This is also something that the multiple linear regression could handle. In the example I gave, the examples were categorical (the presence/absence of a colour) but I could make them discreet, by changing the variable to be "# of red cards" or "# of colorless cards", and so on.
The only other issue is that for the multiple linear regression to work, it assumes that each variable is independent. This means that having white in the deck doesn't affect the performance of the red cards. But sometimes it does, especially with Boros. Anyone ever notice that there are more good mono-red commanders than Boros commanders? To account for this, I would have to add a variable for color pairs as well, which turns a 5 variable equation into a 32 variable equation.
In any case, this is a lot of work that I don't really feel like doing, but all of this is possible to do.
-
In any case, this is a lot of work that I don't really feel like doing, but all of this is possible to do.
I'm actually more interested in your opinion on the matter: do you think this approach would be more appropriate than the one Command Zone took?
The only other issue is that for the multiple linear regression to work, it assumes that each variable is independent. This means that having white in the deck doesn't affect the performance of the red cards. But sometimes it does, especially with Boros. Anyone ever notice that there are more good mono-red commanders than Boros commanders?
This is a very interesting point. Many colours struggle to draw cards or ramp efficiently, which means that cards like Mind Stone (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mind+Stone) are probably more powerful in a blue artifact synergy or Boros deck that struggles to generate extra mana while for example green could easily substitute it with Rampant Growth (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Rampant+Growth) or Llanowar Elves (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Llanowar+Elves). Sometimes you may not have a choice if Mind's Eye (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mind%27s+Eye) is the only draw engine that is available, in which case it's not powerful, it's irreplaceable.
Perhaps the question itself is insane. X (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=X) is the most powerful colour...in a vacuum? I'm pretty sure a generic mono green control deck would be pretty bad. Or a mono white artifact combo. Either way, the answer we're looking for is probably not very descriptive of the actual problem we're trying to solve. For single cards, sure. You can probably measure the impact of Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) and maybe roughly say that it's most effective in artifact combo and least effective in green decks that ramp fast anyway - and that we can generate some numbers that mean something. But a part of the insanity of the question is that Boros shouldn't be worse than mono red, because the latter is a subset of the former.
-
I am going to regret this. I do not want to revive (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Revive) this thread. I am going to regret this. Why am I bringing back this trauma? I am going to regret this.
*Deep inhale*
I'm looking back through this thread to re-gather some of The Command Zone's conclusions. I'm re-doing one of the analyses, that I'm pretty sure the data set will help me solve.
But I also never answered this question.
In any case, this is a lot of work that I don't really feel like doing, but all of this is possible to do.
I'm actually more interested in your opinion on the matter: do you think this approach would be more appropriate than the one Command Zone took?
I think that any approach would have been more appropriate than what The Command Zone did.
They hired a guy (with money) to make some graphs in Microsoft Excel. Anyone with a high school education could have done that. They made the graphs, but didn't do the stats. While the multiple linear regression would take a long time, a correlation test takes a matter of seconds assuming the data is organised properly. A chi-squared test takes 5 minutes, and you can do it on the back of a napkin with a calculator nearby if you really need to. Both of these tests I did several times. Where's my money?
-
I am going to regret this. I do not want to revive (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Revive) this thread. I am going to regret this. Why am I bringing back this trauma? I am going to regret this.
*Deep inhale*
I'm looking back through this thread to re-gather some of The Command Zone's conclusions. I'm re-doing one of the analyses, that I'm pretty sure the data set will help me solve.
But I also never answered this question.
In any case, this is a lot of work that I don't really feel like doing, but all of this is possible to do.
I'm actually more interested in your opinion on the matter: do you think this approach would be more appropriate than the one Command Zone took?
I think that any approach would have been more appropriate than what The Command Zone did.
They hired a guy (with money) to make some graphs in Microsoft Excel. Anyone with a high school education could have done that. They made the graphs, but didn't do the stats. While the multiple linear regression would take a long time, a correlation test takes a matter of seconds assuming the data is organised properly. A chi-squared test takes 5 minutes, and you can do it on the back of a napkin with a calculator nearby if you really need to. Both of these tests I did several times. Where's my money?
Check is in the mail :D
-
I am going to regret this. I do not want to revive (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Revive) this thread. I am going to regret this. Why am I bringing back this trauma? I am going to regret this.
*Deep inhale*
I'm looking back through this thread to re-gather some of The Command Zone's conclusions. I'm re-doing one of the analyses, that I'm pretty sure the data set will help me solve.
But I also never answered this question.
In any case, this is a lot of work that I don't really feel like doing, but all of this is possible to do.
I'm actually more interested in your opinion on the matter: do you think this approach would be more appropriate than the one Command Zone took?
I think that any approach would have been more appropriate than what The Command Zone did.
They hired a guy (with money) to make some graphs in Microsoft Excel. Anyone with a high school education could have done that. They made the graphs, but didn't do the stats. While the multiple linear regression would take a long time, a correlation test takes a matter of seconds assuming the data is organised properly. A chi-squared test takes 5 minutes, and you can do it on the back of a napkin with a calculator nearby if you really need to. Both of these tests I did several times. Where's my money?
While I think it's positive to question the results published by the Command Zone (when having enough knowledge to do so), I think they made clear from the beginning that, while the data base they put together was as wide as possible, it still wasn't enough to be mathematically accurate.
So I don't really see the point in reiterating the same statement.
Also, I don't see the point in claiming one can do more accurate stats on the same data base that we all kind of agreed isn't accurate itself.
-
You're completely missing the point. Aside from a few of the games, there is nothing wrong with the data set. It's enormous, its well recorded, and it represents a good sample of games played by YouTubers.
The data is fine, the stats part is not. You can't just make a bar graph and conclude that an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) causes you to lose more often than not. That is not good statistics. That just spreads misinformation (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Misinformation).
-
I don't want to sound arrogant, but if I understand correctly we're talking about something more than 300 games.
I mean, EDHREC has more than 200,000 decks, this site has 10,000 decks, what do you pretend 300 games could represent, compared to these numbers?
-
300 games gives us actual data on how games of commander operate. A decklist tells us nothing about what the strongest opening moves are, the importance of interaction, or which colors are the strongest in commander. It doesn't matter if you have 200 000 decks at your disposal, you need to see them in action. They need to be played against other deck, and data needs to be gathered on how they performed.
That's what the data from The Command Zone is good for. It's actual recorded games, with a wide variety of decks, playstyles, and players. Considering that for most statistical analyses a sample size of 30 or 50 is enough, 300 is an excellent sample to work with.
The downside is that it is not a sample of all possible commander games, it is just the population of commander games posted on YouTube. But there are still inferences that can be made from this. I can say with confidence that an early game Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring)/Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) does not correlate with losing.
Next I'm checking turn order. Does going first make you more likely to win?
-
I insist (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Insist).
Assuming each of the 300 or so games were among 4 players, we are representing at most 1200 decks.
To give an idea, this is 1 order of magnitude less than the decks in this website; add the fact that the decks involved in said videos are often quite random (like, all commanders from the newcoming set, all tribal decks, all budget decks etc.), this by no means can be considered to represent the metagame.
I mean, you don't need statistics to know that going first or playing an early Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) helps you win the game.
The very fact that they had a Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring)'s win-rate slightly negative (which is at the very least counter-intuitive) only confirms my point: the data base were insufficient.
Their "fault" only has been to set aside this handicap and try to explain the outcome by saying people try to team up against the player who plays an early Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring), which by the way ain't that out of this world as a sentence...
Moreover, I'm not undermining your calculations, I'm just saying you probably went too far with them. Only that.
-
I don't want to sound rude, but you're wrong, crimsonking. 300 games is a whole lot more data than 10 000 decks.
Consider this: most studies in psychology have way smaller sample sizes with far fewer variables. They don't usually struggle to achieve p<0.05, which is considered 'significant enough'. Like Morganator said this data set is very representative of a typical Youtuber game.
300 doesn't sound like a lot but many relationships between variables start to show up already at n=20. I'm very surprised if the data set is somehow ambiguous and number crunching ends up saying "we have no clue".
TL;DR A data set of n=300 is plenty.
EDIT:
Also pretty sure they did a shitty job with the stats. Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) should come out on top if you look at it game-by-game instead of comparing averages to averages. You'd be surprised how weird results one can conjure up with Bad Math (™).
-
I could see a case to be made for a t1 Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) making you an early target due to a fast start. The same could probably be said about Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) and the like. That is, assuming how fast a start it gives you and whether or not the table can catch (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Catch) up.
-
I could see a case to be made for a t1 Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) making you an early target due to a fast start. The same could probably be said about Mana Crypt (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mana+Crypt) and the like. That is, assuming how fast a start it gives you and whether or not the table can catch (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Catch) up.
They also said if you don't capitalize by turn 3, turn 1 Sol Ring (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Sol+Ring) ain't that effective as expected, which in my opinion is totally true.
In fact, if you're going to get advantage from it on turn 4 or later, I'd say Cultivate (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Cultivate) could be a better option.
This makes sense in a casual game, but it's obviously bulshit in cEDH.
What I'm trying to explain all along is that statistics is a nice tool as long as you have a sample that represents your population.
I'm a senior software engineer and you can mark my word when I say you can statistically prove whatever you want by purposefully selecting your sample data (I'm not saying the Command Zone did that, though), especially in psychology! ;D
-
I'm a senior software engineer and you can mark my word when I say you can statistically prove whatever you want by purposefully selecting your sample data (I'm not saying the Command Zone did that, though), especially in psychology!
But we're not purposefully selecting sample data, and I have been fully open on what the population is. I have data from 300+ games based on YouTubers. This sample represents the population of possible games among YouTubers.
I'm not biased one way or the other, and I'm not manipulating statistics to suit my needs/desires.
-
Do you know what my favorite thing about Deckstats is? The stats.
That joke will never get old.
Null hypothesis: The turn order has no significant effect on the outcome of the game. Everyone has equal probability of winning no matter what turn they start on.
Alternate hypothesis: The turn order does have an effect on the outcome of the game. If the alternate turns out to be true, I will test to see which turn is the best and which is the worst.
The first part was organizing the data so it makes sense. It's divided into two categories. Observed is what actually happened. Expected is what we assume would happen if the null hypothesis is true. If what we observed does not match the null, we accept our alternate hypothesis.
| | | First | | Second | | Third | | Fourth | | Total |
| Observed | 92 | 70 | 73 | 68 | 303 |
| Expected | 75.75 | 75.75 | 75.75 | 75.75 | |
Just so its clear, the 92 means that 92 of the winners went first, 70 of the winners went second, and so on. Now its very easy to jump (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Jump) to conclusions and say that going first is best (this is what The Command Zone did by the way). After all, the people that went first won more games than anyone else. But you always do the math, just to make sure. Chi-squared time!
I keep talking about this test, so I should probably go into more detail about it. Chi-squared checks to see if the result you got is not what you expect. If your observed values are out of the ordinary, you reject the null hypothesis. It uses this formula.
(https://www.statisticshowto.datasciencecentral.com/wp-content/uploads/2013/09/chi-square-formula.jpg)
That really big letter "E" looking thing means "the sum of all". Oi is a given observed value and Ei is it's respective expected value. So in our case:
(https://i.imgur.com/Z9YHyG7.jpg)
This gives us our chi-squared statistic of 4.81, which means nothing without context. Next you compare it to a Chi-squared table, and see if this chi-squared statistic exceeds the value on the table, with our given degrees of freedom and significance.
Shoot, forgot to mention that. Degrees of freedom is always n-1 when dealing with a sample. Because we have four outcomes for who is going to win (first player, second player, third player, and fourth player) we have 3 degrees of freedom. Our significance value is 0.05, which more or less (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=More+or+Less) means "There is a 5% chance that we got this result through luck". Here's the table.
(https://i.stack.imgur.com/PbEqv.jpg)
With 3 degrees if freedom and a significance of 0.05, the value we are trying to exceed is 7.81... which we did not exceed. We only barely got past the 0.25 significance value, which is not enough to reject the null.
Conclusion: going first does not give a significant advantage over the other players. There are just way too many other factors that come into a game for turn order to be the deciding factor.
-
You have to eliminate as many other variables as you can to determine if turn order does anything.. hence my suggestion to play games where all the decks are the same
-
Conclusion: going first does not give a significant advantage over the other players. There are just way too many other factors that come into a game for turn order to be the deciding factor.
I said this to begin with awhile back ;)
-
I said this to begin with awhile back ;)
Yep, you were right.
You have to eliminate as many other variables as you can to determine if turn order does anything.. hence my suggestion to play games where all the decks are the same
I would really like to eliminate all other variables, but I can't see any way of doing this where it represents a good population of commander games. What the analysis of The Command Zone's data shows is that turn order is not a strong factor, so we shouldn't worry too much about it.
-
Their data definitely didn't isolate (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Isolate) any variables
-
I'm a senior software engineer and you can mark my word when I say you can statistically prove whatever you want by purposefully selecting your sample data (I'm not saying the Command Zone did that, though), especially in psychology!
But we're not purposefully selecting sample data, and I have been fully open on what the population is. I have data from 300+ games based on YouTubers. This sample represents the population of possible games among YouTubers.
I'm not biased one way or the other, and I'm not manipulating statistics to suit my needs/desires.
Didn't you point out at somewhere in this thread that the games were all meant to be entertaining? And as you mentioned, watching people combo out isn't exactly fun or entertaining to watch. This is why combat damage was the deciding factor in most games. Isn't that some bias in the sample as they are all youtubers?
These are YouTube videos, they are being done for entertainment. That's why combat damage was the most common win condition; watching someone win with a combo is boring, but with combat damage, it's more entertaining to see someone pull ahead. That's why Helix Pinnacle (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Helix+Pinnacle) is a more common win condition than Approach of the Second Sun (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Approach+of+the+Second+Sun); it's more entertaining to watch. Those 304 games aren't a sample of all possible commander games, they are only a sample of the possible games made by YouTubers.
I mean, I know it isn't purposefully selecting the sample as it is the only decently sized sample we have. Perhaps we could all keep track of our stats as Wwolfe and Morganator are so good at doing and compile our own, non youtuber sample set.
Okay, this isn't really stats, but I have been wondering about this. Some dude or collection of dudes (and dudetts) was able to figure out exactly how many chess games were possible. They set the parameter that each player is always trying to make the best possible in order to win, so to exclude (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Exclude) games where each player moves their knights back and forth between the same two spots, creating infinitely many games. Here is the video. Very interesting. I liked it.
https://www.youtube.com/watch?v=Km024eldY1A
Is it possible to do this with MTG? I mean we would need more parameters. For instance you can't have an infinite number of players with infinite deck sizes. Perhaps for 1v1 60 card decks and for 4 person 100 (well 99) card commander decks. I am no mathematician, so I am not 100% positive, but would it be reasonable to assume that since no actual parameter has infinite options, that there is a finite, quantifiable amount of possible games of Magic: The Gathering?
-
Highly unlikely, you see in chess there are only so many moves you can make and you're only ever playing against a single opponent (no players are removed from the game until a loss because there are only ever two players to start with) and both players start with the exact same pieces. If I'm not mistaken I believe that there was a study done showing that the player that goes first in chess has a slightly higher chance of winning a match because they can make more active moves than reactive ones (i.e. the best defense is a good offense strategy), but I could be wrong about that. Anyway, the point I'm making is that unlike in chess MTG has nearly infinite (if not infinite) possibilities due to the vast number of "pieces" (i.e. cards), combinations of cards, randomness and a much wider range of variables. You could make the "best move" assumption but again there are too many variables from game to game for that to be even somewhat practical because the best move for a blue deck will be vastly different from a green deck (assuming mono colors for simplicity sake) and in a multiplayer game those choices can have vastly different outcomes due to a wide range of variables; is there a red deck that wants the green deck to take out the blue deck? if so how might that other player react to what the blue player does in response to the green player attacking them for lethal? how many players can make responses and how many times can they respond in a single turn? will those responses negatively impact them in the long run or will they have a positive effect on the outcome for that player?
those are just a few examples of variables or questions about variables that have to be considered in MTG where as in chess there are obvious best moves to make for both players in any given situation. There are fewer variables in chess and much fewer outcomes and possibilities (granted there are still a crap ton of them).
I also want to point out that I'm not an expert in math of any kind (in fact I'm quite terrible at anything beyond the most basic of multiplication, division, addition and subtraction (I can do Pythagorean theorem and order of operations and that's about it)) so don't expect me to do any calculations, but I felt that this was probably just a common sense answer to the question (more or less (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=More+or+Less)) and I'm not saying that it can't be done with 100% certainty, I'm just saying that the likelihood of it being within the realm of possibility is insanely insignificant (like probably will never be possible). Its essentially a non-possibility because there are just too many variables unless you're basically a god or something.
-
Okay, this isn't really stats, but I have been wondering about this. Some dude or collection of dudes (and dudetts) was able to figure out exactly how many chess games were possible.
Is it possible to do this with MTG?
Sort of, but not really. You are right in saying that there are only a finite number of games that can be played if you exclude (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Exclude) conditions that repeat, loop or recur. It's perfectly ok to make that assumption, because long sequences of repetitive actions can be shortcut, usually leading to a game that ends. There might be a few fringe strategies that rely on getting the deck in a particular order via shuffling but those are usually frowned upon and either lead to a draw or a loss due to slow play.
The problem here is magnitude. While it's possible to tie up all the loose ends through rulings you can't really do anything about randomness in shuffling. Players don't have complete information about the game (unlike in chess) so they have to make assumption about the best possible play. Defining said best play is a little bit hard. Let me illustrate:
- I'll make the best play based on all games I've ever played using the previous games as an average
- I'll make the best play based on all games I've ever played with this format, using previous games as an average
- I'll make the best play based on all games I've ever played against this deck arcetype
And so on. So while it's possible to tweak the rules so that nothing is infinite* you still can't grant players full information about the game. Now, you could say that decisions must be based on your knowledge of Magic alone, in which case you can get a finite set of criteria (because you've only played a finite number of games in your lifetime) but ultimately you can't do anything about the fact that often times in multiplayer for example we assess someone to be a higher threat due to their intelligence. That's not a criterion you could conclude from any finite set of anything, really.
Now here's the asterisk. There are also a few cards that pose a real problem: cards that force you to do something that is random and you can't choose to stop doing so. In an infinite loop either you, the opponent(s) or the judge usually have a chance to react somehow and the game ends. However, there are four cards that aren't guaranteed to ever resolve: Crazed Firecat (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Crazed+Firecat), Mirror March (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Mirror+March), Okaun, Eye of Chaos (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Okaun%2C+Eye+of+Chaos) and Zndrsplt, Eye of Wisdom (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Zndrsplt%2C+Eye+of+Wisdom). If you win every flip you have to keep flipping. These cards are truly infinite in the sense that there is no defined outcome, you cannot shortcut them and the rules don't allow you to exit the situation. The opponent could scoop but I don't know what the rules are for scooping or if you can ever justify conceding to be "best possible play". To me it sounds like the exact opposite. It'd also have to be 'faster' than instant speed because you're doing it during the resolution of another card and I'm not sure if the rules even recognise such a state. Except Panglacial Wurm (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Panglacial+Wurm). Panglacial Wurm (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=Panglacial+Wurm) is always fun at parties...
TL;DR
No. Rules, cards and criteria for decision making are too ill defined.
-
Didn't you point out at somewhere in this thread that the games were all meant to be entertaining? And as you mentioned, watching people combo out isn't exactly fun or entertaining to watch. This is why combat damage was the deciding factor in most games. Isn't that some bias in the sample as they are all youtubers?
There is no bias in the sample, because it's taken from the population of games with YouTubers, which happens to have a low amount of combo decks.
-
That's generally a "bias" that is found throughout the entire format.. cEDH is unfortunately a minority
-
If I'm not mistaken I believe that there was a study done showing that the player that goes first in chess has a slightly higher chance of winning a match because they can make more active moves than reactive ones (i.e. the best defense is a good offense strategy), but I could be wrong about that.
Chess is not a good reference for MTG in any format . Gary Kasparov begs to differ with your quote :-) . He redefined c4 and it still doesn"t have an answer almost 40 years later . I just do not think the 2 will ever be compatible . But I do enjoy reading the responses ! :-D
-
the whole chess analogy was because the person I was responding to was using it as a basis for a question. I was literally explaining why the two are incompatible on the basis of their question. Because the two are nothing alike. You can't really compare any card game to chess because while you can calculate odds and probabilities for card games you won't always be correct because your using assumed information with a best guess backed up by probability whereas in chess all information is known to both players at any given time (more or less (https://cards.deckstats.net/magiccard.php?utf8=1&lng=en&card=More+or+Less)) and there is a best possible move in every situation, granted that just because you make the best possible move doesn't mean you will always win. Chess is a game of skill and strategy and cards games (even games like MTG) are games of chance with some skill and strategy involved, but it really comes down to luck because that can make or break you in any game you play and luck can not be calculated in a mathematical formula.