Some numbers on how much each side wins

[SurlyBen]

A few days ago I decided to see just how much more often the axis won than the allies. So I took a pool of 769 battles fought in the Band of Brothers combat mission club, and I ran 'em through microsoft excel.

I posted the stats to this thread over in the Band of Brothers forum, and wandered off. Anyway, along comes a post over here about how the Americans are completely favored by CM, which made me decide to post 'em here to. For the time being, we'll ignore that said post appears to have been a troll to stir up trouble. (I was a bit hesitant to post this for the same reason...)

To start with a few caveats:

My numbers may be wrong. I checked 'em, but I never took that statistics class in college, and I have already noticed (and fixed) at least one huge mistake. If you notice any mathematical errors, feel free to let me know...

My numbers may mean nothing. They don't distinguish between human picked QBs, computer picked QBs, games played under various force type settings (armor, mechanized, infantry only, etc...) scenarios, games played for fun, games played to win, and games played under special rules for picking units (Fionn's short 75 rules for example). A lot of this data is available, if anyone is interested enough to get it.

I have no idea what the margin for error is. If you know, feel free to let us all know

Finally, these numbers clearly show that it is possible to win with any side at any skill level.

I looked at overall allied, german, british, and american forces. For Polish, Canadian, and French there weren't enough battles to get a meaningful result (17, 9, and 13 fights respectively)

The percentages and point values are all rounded off to the nearest tenth.

On to the stats:

The average axis score is 50.2

the average allied score is 44.7

The average american score is 42

The average british score is 50.7

Overall percent of games won by either side (including unranked players):

Allies win: 38.9%

Germans win: 49.5%

Draws: 11.6%

For the americans (out of 357 battles)

Win: 31.9%

Lose: 54.6%

Draw: 13.5%

For the brits (out of 211 battles)

Win: 49.3%

Lose: 40.3%

Draw: 10.4%

Then I decided to look at how the scores changed based on ranking on the ladder.

Players ranked higher than 100 (there are 29 players)

Allies(out of 292 games)

Win: 55.1%

Lose: 30.8%

Draw: 14.0%

Germans(out of 279 games)

Win: 72.0% (!)

Lose: 17.9%

Draw: 10.0%

Players ranked from 90 to less than 100 (26 players)

Allies (out of 186 games)

win: 37.6%

lose: 50%

draw: 12.4%

Germans (out of 224 games)

win 44.6%

lose: 42.0%

draw: 13.4%

Players ranked below 90 (30 players)

Allies (out of 239 games)

win: 23.4%

lose: 67.4%

draw: 9.2%

Germans (out of 210 games)

win: 29.5%

lose: 59.5%

draw: 11.0%

So there you go... My guess is that you already knew all of this The only surprising thing to me was the win percentage for the british.

Surlyben

[ 08-30-2001: Message edited by: SurlyBen ]

[Slapdragon]

We tried this about a year ago, but I cannot find the thread. The averages of Rugged Defense and Grognard ladders for CM had allies overall weaker by a small but significant amount, but the British actually being stronger than the Americans. Experienced players benifitted the most by being on the defense and by being German. Results were all QBs. We had stats that showed all of this was significant.

Still, with the patches and the tweeks, I could not say who was on top now. I assume even since my last games seemed even. It is at least close enough that looking at it from a quantitative perspective makes the game look really well balanced.

[lcm1947]

Well, doesn't that show that the German's win over the American's most of the time? Just asking.

[Doug Beman]

lcm, yeah the numbers seem to indicate that the German side wins more often. That's why the thread was posted here; there's been a few threads recently about how fighting in CM favors the Allies.

These recent posts are interesting when compared to past threads which stated (some quite vehemently and angrily) that CM is tilted strongly in favor of the Germans.

DjB

[Ron]

Interesting stuff in a sense, but like you commented, it would be more useful if more of the 'parameters' were known and broken down. I would be especially interested in the types of battles being fought by the corresponding sides that achieved the win/loss ratios you concluded.

As for the win percentage of the British, that doesn't suprise me. They may not have uber-infantry but the Firefly is in a class by itself when it comes to armor battles as portrayed in CM, and their flexible artillery more than makes up for their 'weak' infantry. I remember early on in the first Rugged Defense Tournament, say about 60 games finished, the win/loss for the Brits and Germans were basically dead even.

Ron

[CombinedArms]

These numbers look significant to me. Since the US has pretty good infantry squads and good artillery, but highly vulnerable tanks with weak guns (while the British have some heavy tanks and some good tank guns--not always on the same tank), I wonder if what this shows is the overarching importance of tanks? I mean, you can win with the US forces, but it's HARDER--and tanks may be the key difference. Admittedly, it's hard to sort out without better-sifted info, but my guess is that this difference would show up however you sift it.

[Moon]

Stats can be a very dangerous thing, especially when you accept them as facts but fail to see the background. For example - could it simply be that "better" players (i.e. more experienced wargamers, grogs, current military personnel etc.) tend to pick other forces than US for their games more often than - pardon the expression - newbies? Just a thought...

[Vanir Ausf B]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Moon:

For example - could it simply be that "better" players (i.e. more experienced wargamers, grogs, current military personnel etc.) tend to pick other forces than US for their games more often than - pardon the expression - newbies? Just a thought...<HR></BLOCKQUOTE>

Not impossible. Yet, I can't think of any reason why this would be so. In fact, I would think more newbies would go for the Germans because of the "cool cats".

[Enoch]

If you really wanted to know how well the various allies fared against the Germans you would need do an analysis that controls for many of the factors that others have already brought up: The quality of the two opponents facing each other, which side is attacking or defending and that attck defend ratio, infantry only, how much armor, time of day. There could be more factors I can't think of that might effect the apparent results we see.

After controlling for these factors (and possibly others) then you could get a good idea as to whether one side holds a significant advantage over the other.

If I had to guess I'd say that the quality of the opponents and the extent to which one side might be used more in attack or defense.

Finally, a topic I know way too much about.

[Pzman]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Doug Beman:

lcm, yeah the numbers seem to indicate that the German side wins more often. That's why the thread was posted here; there's been a few threads recently about how fighting in CM favors the Allies.

These recent posts are interesting when compared to past threads which stated (some quite vehemently and angrily) that CM is tilted strongly in favor of the Germans.

DjB<HR></BLOCKQUOTE>

Are we all forgeting that the KIA/MIA ratio was higher for the allies then the Germans? From the stuff I have read allied forces toke over 60% of all KIA/MIA (Russians included). Not to say that the Germans were better but think about the olds against them... they held off over 4 enemy conutrys for over 4 years... how many other conutrys the size of Germany have held back some of the largest populated nations of the world for that long?

[ 08-31-2001: Message edited by: Panzerman ]

[Slapdragon]

There is a way to control for a larger group of variables, but all non controlled variables would have to be random. So no scenarios, only QB. Attack and defense assigned random. Either everything random, or picked units by the player (which removes a chance that faulty random equipment purchases skew game). A larger number of games. 100 people each play 20 games for a total of 1000 results. Some categories we control for will be very small, but we check significance on each.

Results would need to be 95/5 in the case of each variable to be assured, but lower confidence could yield weaker results.

By the end of

[xerxes]

All factors DO NOT have to be controlled for. In fact, having a large number of randomized factors and still have statistical signficance allows much stronger and broader real-world conclusions.

The significant problem would be bias. For example, if germans play as the attacker a higher % of the time. A more complete regression model could get at a lot of this.

Given the data, it certainly looks like the germans have a fairly significant advantage. Send me the spreadsheet and I'll give you a statistical analysis and post it.(marcsullivan@excite.com)

Could be due to a number of factors but the germans have highly "adjustable" infantry. Depending on what you need, you can get short-range or long-range infantry, you can inf. with or without shrecks, infantry with or without support weapons. Given the importance of infantry in CM it is not surprising the germans come out on top. General armor strength probably helps too. The improved allied arty probably doesn't compensate enough.

I'd be interested in the results if games of mixed (reg&paras) allies were looked at separately.

Too bad you can't run CM as comp vs comp, you could do a good monte carlo study then.

-marc

[Andrew H.]

I tend to believe that these statistics probably mean that the Germans have an advantage under some circumstances (which may or may not be circumstances under which I or others play). As has been pointed out, these circumstances could be extrinsic to the game -- perhaps it's easier for new players to play Germans.

But it's important to keep in mind that the only thing that these stats mean (assuming that they are meaningful) is that the point values for units might need to be tweaked.

That is, changes such as weakening the front armor of the Jumbo or increasing the armor on the Tiger's mantlet can only affect play balance if the point value of the Tiger or Jumbo does not accurately reflect the value of the vehicles. There would be no effect on play balance if the units were properly priced.

My theory is that CM point values are probably pretty correct for all possible CM games, but that they may favor the Germans for the types of games played by most CM players. Say, 1500 point MEs in clear,dry weather, with small hills and moderate trees.

[Enoch]

xerxes, I certainly wasn't trying to say that you should control for everything. I should have been more clear. You do need to control for things that have an influence on the outcome though. If you don't control for these then you have possible model mispecification. The opposite is also true, if you try and control for to many things or things that aren't important you can have the same problem.

If I had to guess I'd say there are probably 5-8 things (most have been mentioned at some point in this thread) you should control for before concluding there is a significant difference in how each side fares.

[Diceman]

Before Battlefront fixed the "bug", you used to be able to produce mirrored battles. I.E. a player would play two games against their apponent, one axis, one allied, on the exact same map. Never split a game. I'd either loose both, or win both. Man I wish we could incert maps into quick battles!

Each country has their advantages. Cut the practical range down to less than 500 yards, Allied armor can rule the day if its advantages are used properly. Cut infantry ranges to less than 200 yards but over 50 yards U.S. squads rule. At less than 100 yard ranges certain German squads truely rock.

You can seriously scew a game in favor of one country or another just on the settup. All the possible variables make it impossible to say one side is better than another unless you narrow things down to more specific situations.

[Redwolf]

My two cents:

The British probably look better because inexperienced users usually choose American. Also they have a very robust and slow tank which may not be suitable for blazing total victories, but you probably see the statistics reflect that this thing doesn't usually get K.O.ed in the first minutes.

The Americans work (IMHO) best with a firepower play style, but if you run out of ammo, you are in more trouble than with other nations. While this will not usually happen to most players, considering so many games in a statistics you will see the a number of them dropping the average.

The German have the advatage of a greater choice of tools. In a big-base statistics that will show more clearly than in the single games.

Also, when the battle is chaotic and infantry/armour coordination falls apart, the Germans have the edge because insolated infantry-vs-infantry and isolated tank-vs-tank they are better. While you may not see that many battles of this kind if you follow single battles, it will still raise a number in a statistic like this.

[FFE]

<BLOCKQUOTE>quote:</font><HR>Originally posted by SurlyBen:

The only surprising thing to me was the win percentage for the british.<HR></BLOCKQUOTE>

First let me write, thank you for compiling this information. It has been my belief, ever since I engaged in Case's SMG ladder several years ago, that the true balance of any game can only be measured by competitive play. Casual playing tends to lead to casual results. While ladder games might constitute the worse aspect of game play, it unquestionably displays significant trends that can directly link game balance issues.

With that in mind and since I am a veteran of CM ladder games, there's a good reason the British win % is significantly higher. It's called, Churchill VII/VIII. The Axis player, tending to be fixated with Hetzers and Panzer IV/70's, has a bear of a time trying to take on the British Behemoth. Other reasons the British tend to fair well; 3" mortars, Paratroopers, and Wasps.

As a trend, I agree it's easier to win as the Axis. Most conflicts are played on a limiting map: the Axis side has more instruments to deal with whatever map is presented. Personal statistics indicate I tend to win more often with the Axis than the Allied. Although, my personal preference is to play the Allied side.

[Vanir Ausf B]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Slapdragon:

By the end of<HR></BLOCKQUOTE>

Somebody go check on Slappy. Something appears to have happened to him mid-post.

<BLOCKQUOTE>quote:</font><HR>Andrew Hedges:

But it's important to keep in mind that the only thing that these stats mean (assuming that they are meaningful) is that the point values for units might need to be tweaked.<HR></BLOCKQUOTE>

Very good point. It doesn't necessarily mean the German units perform better than they should, but rather that they are too cheap for their performance.

[JasonC]

One fellow made the comment that the definite statistical edge the Germans show must reflect prices that are somewhat off. I think that is right; more on it below. But first a few other minor issues others have raised.

One fellow suggested perhaps Germans are just easier to play for new players. It don't think that is consistent with the data; it is backwards. The differences are largest for the most experienced players, and almost disappearing for the lowest ranked ones. Lack of skill can hurt either. But when skill is present to exploit the German advantages, that is the place you see the huge win figure.

Another fellow suggested, in effect, the Germans should win more, and brought in the Russians as part of his reasoning why. This is obviously off for three reasons. One, point costs are meant purely for balance, not to reflect historical costs or anything. If equal points produce different fighting power, they have not balanced things. Two, there aren't any Russians in CM today. Just the western allies, and in NW Europe, and in the last year of the war. Which in case anybody forgot, was a time and place the Germans didn't exactly do terribly well. And they lost most of the strength of OB West twice over. Naturally, because they were facing superior odds. And three, German losses in the west were above those of the allies. (They lost more in Normandy alone than the US did in the whole war, e.g.). Which doesn't mean the allies should win more often, because balance is the point of unit costs. Just that the reason given is off.

The areas of pricing that obviously benefit the Germans are -

1. high ROF infantry weapons are particularly effective for the cost. Their cost and their ammo do not fully reflect their value and rate of burning the stuff. The average German squad has more automatics than the best Allied ones, and some have nothing but.

2. tank dueling ability is underpriced compared to other vehicle extras, which by comparison are unimportant. A better gun and better front armor are cheap for their battlefield effect; faster turrets, 3rd MGs, minor ammo load or marginal armor differences are expensive. Thus improved Shermans cost as much as Tigers that outclass them in everything important, while Hezters are cheaper than Stuarts, and Jadgpanzers are the price of Sherman 75s.

3. The cheapest direct fire gun types have disproportionate combat power for there low cost. Those with prices up in the 50-100 range are often fairly valued, but guns with prices under 40 are bargains. And the Germans have large numbers of these. 50mm ATGs cost 2/3rds what 57mm ATGs cost. 75mm infantry guns cost as much as squads, but can decimate a whole platoon in a few minutes. 20mm FLAK are cheaper than MG teams but far more lethal. Puppchens cost only about as much as schrecks but have 2-3 times the effective range and more shots. Meanwhile, the Allies don't have a single gun below 50 pts. The counter, light mortars, in general have too little ammo to produce such impressive combat effects, while costing as much as the guns they might counter. The Brit 3" is the only comparable bargain on the Allied side - and the Brits show better performance.

4. Artillery is both expensive, and the most restricted force category in most force types. Historically it was superiority in this arm - mostly due to ammo supply, some to technique and numbers, and also including air power - that helped counter German strengths in maneuver arms. There are few fights Allies lose that wouldn't be turned around by 9 105mm FOs firing 300 rounds apiece. The indirect guns they Allies favored are fully priced to expensive, "lumpy", and in the most pinched category. The lighter direct guns the Germans made more extensive use of are cheap, buyable in any amount, and in the abundant "support" category.

There are some other mispricings, but in areas that do not make much difference for game outcomes. Thus, foot flame teams are overpriced and demo charges are underpowered, but it makes little difference. Carrying ability is overpriced for the vehicle category, while fighting power and ammo are underpriced. But M3A1 scout cars (e.g) will not win a battle for you alone.

The higher German win rate reflects the ability of players to spot and exploit these pricing issues. By bending battles far from vanilla situations, the smallest weakness of the pricing system are amplified. Differences that would be marginal of both sides have plain, common forces, can be made game-deciding by only using the rare types that exploit the discepencies the most. Thus there are more German automatics, fwer rifles, in CM QBs than in history; there are hordes of light guns beyond any realism; the less armored half of the German armor force is AWOL, etc.

The above is just diagnosis, not prescription. Of course BTS has acknowledged some of these issues, and hopes to address them with rarity system options for CM2. In the meantime, some people use force restrictions (Fionn 75, "no SMG hordes", etc). And I suggest ammo tweaks to at least partially address the first of the issues - the effectiveness of infantry automatics for their cost.

[Wilhammer]

JasonC,

That is the best explanation of the cost bias (unintentional) that favors the Germans I have read.

The Sherman thing is particularly way off, and your other arguments are right on (with me).

[Andrew H.]

I think that Jason's examples are all good, although I believe that the most significant pricing advantage that the Germans have is in the area of armor. As I understand it, CM uses a formula that takes into account things like armor, AT capability, anti-infantry capability, speed, number of rounds carried, turret speed, and I'm sure several other factors. Previous posts by BTS have noted that the Sherman's significant anti-infantry capability makes it more expensive than its armor and AT capability (including turret speed, etc.) would suggest.

The difficulty in typical games, however, is that the Sherman's anti-infantry capability is contingent on the Sherman being able to survive attacks from other tanks (and, to a lesser extent, AT guns, PFs, Schrecks,etc.). Because the anti-infantry properties of the Sherm are contingent upon the tank surviving at least a couple of turns, the anti-infantry value for the Sherm should be discounted based on some probability of the Sherman being able to employ its anti-infantry (AI) capability. In large part this discount should reflect survivability, but to a lesser extent it should probably reflect the fact that the tank might not be able to shoot up as much infantry as it would like because it has to skulk around to avoid being killed.

However, arriving at the proper discount is not easy because there is no one proper discount. As I suggested in my earlier post, BTS might have the value right for the entire universe of CMBO, but most people don't play the entire universe of CMBO. For example, I don't think that there are many ladder games involving combined arms attacks on infantry defenders, even though this was probably the most common type of action in France '44, and, incidentally, a type of battle in which the Sherman's AI capacity would have a high likelihood of being used.

So the real solution would be to adjust the value of units based on the possibility of certain other units being present or absent. Which would be pretty complicated, I'll admit.

Incidentally, the fact that the British appear to do about as well as the Germans provides more evidence for the tank supremacy theory, as the British have two tanks -- the Churchill and the Firefly -- that are very effective in the CM universe.

I do find that Fionn's rules work very well at balancing things out, not only because they prevent ueber-battles, but because the point values seem more accurate in the Fionn universe -- there is a better chance of the Sherman living to use its AI capacity in the Short-75 world than there is in the entire CM world.