Steel Beasts vs Combat Mission t-72 visibility test

[domfluff]

Ah, gotcha - "CM doesn't model any of that (doctrine)" is "CM doesn't model the differences in radio use", rather than "There's no difference in doctrinal behaviour between factions in CM".

Yes, I believe this is correct. the C2 model is very sophisticated, and more complex than possibly anything else out there, but it's not quite *that* sophisticated.

I suspect it doesn't do a bad job of faking it though - particularly in the modern titles with radios everywhere, and various battlefield management systems.

[holoween]

1 minute ago, domfluff said:

I suspect it doesn't do a bad job of faking it though - particularly in the modern titles with radios everywhere, and various battlefield management systems.

Once you notice it it becomes very obvious and in the modern titles id say it matter more.

Just ran a test with a german tank btl and information about a hostile tank was at the btls hq before it was at the platoonmates tank which is simply impossible irl given how the radios are set up. It still got there in a minute but it should have only taken a few seconds.

[A Canadian Cat]

2 hours ago, Vanir Ausf B said:

I was looking for that - thanks! It is really interesting.

[A Canadian Cat]

1 hour ago, holoween said:

It doesnt model things like an entire platoon being on the same net so as soon as information gets transmitted over the platoon net it should reach everyone but ingame it only reaches the platoon leader.

Well that last part is not correct. The rest of the platoon does get the information. I suppose you are not wrong about who gets it first and who gets it next. Adding that extra level would be pretty cool but there is more to doctrine that radio nets. I maintain that most of units behaving according to doctrine is really up to us (and scenario designers) when we issue orders.

[akd]

Would be interesting to test speed of lateral contact propagation versus vertical propagation, but these are again things that require careful controls and large sample sizes.

[Centurian52]

11 hours ago, The_Capt said:

I would be interested to see what the T80 can do.

One more time with the T80B. Surprisingly it did significantly worse than either the T64A or the M60A1. I suspect the issue is that 10 tests is too small of a sample size and it just had a run of bad luck. Or maybe it actually does have worse spotting.

T80B spotting M60 at 2k

(?) spot     firm spot   (in seconds)
32             39
207           221
59             80
99             106
252           259
75             82
115            115
84             91
173           173
94            102

average for (?) spot: 119 seconds

average for firm spot: 126.8 seconds

[dbsapp]

Below is CIA assessment of comparative performance of Soviet and US tanks capabilities. 

As you see, CIA concluded that Soviet tanks have quality advantage and made a model that showed that in tank duels they will win. It took into consideration data on fire rates and accuracy.

I never saw in any real life documents any mention of Soviet tanks having daylight optics problem. 

The doc is declassified and can be downloaded  from CIA website. 

https://www.cia.gov/readingroom/document/0000624298

 

Edited October 21, 2021 by dbsapp

[BeondTheGrave]

Lending credibility to this document, Paul Gorman is legit. He was a TRADOC guy all the way back to DePuy's tenure in the mid-70s, so hed had over five years working on solutions to this very problem. I'd argue hes was a bit more willing to.... instrumentalize his analyses for the sake of pushing an agenda (usually based on the gear he was hoping to get the Army to acquire), but overall pretty good. 

The analysis, though, like most US analyses, overrates the T-72 in glaring ways. Take, for example, the fact that on pg. 12 he bases part of his conclusion on the fact that the T-72 had a laser rangefinder. As weve discussed, it didn't. It used coincidence, less quick and accurate. The information you cite from page 14 is something you see pretty commonly in these documents. "Oh no! The T-72 is 63% better than the M60A3! See my numbers!" Except that much of that assessment is based on soft factors which are nearly impossible to quantify, and even if they were almost always tend to overrate strengths and ignore or downplay weaknesses (because, I'd argue, theyre sales pitches. But thats another conversation entirely). 

This document, I think, is really trying to get at what Gorman thinks is "faulty intelligence" regarding the capability of the T-72 and the potential detrimental effects it might have had on US R&D at the time. Hes especially concerned that if the armor and strength of the T-72 had not been revised upwards, the US may not have proceeded with the "XM1-E1" upgrade program (we would know it as the M1A1) or the I-TOW, and instead focused on the Dragon and bringing the M60s up to snuff. I would suspect, though I dont know for sure, that this has something to do with the sudden realization in the west that the T-64, -72, and -80 were indeed all separate designs of progressing quality. For many years the T-64, for example, was called the T-70 and the US was a bit unsure as to what its relationship was to the T-72. This only cleared up around the late 70s, though that hadn't filtered down via TRADOC training documents. I have an old edition of FM 100-2-3, think its from 1980, that calls the T-64 the T-70 and only hypothesizes about better armor (if memory serves). 

All this is to say its an interesting document, but being used out of proper context here. The US didn't know at the time what the later T-series tanks were all about, nor did they have a good grasp on equipment and capabilities. Keep in mind the last tank the US had gotten a hold of were T-62s from the YK war. I dont even think the T-72 came westward until Desert Storm. This led to an analogous to WWII "Tiger mania," T-72 mania. Desert Storm really shattered that illusion because so many officers had grown up in the 70s and 80s fretting about the T-72 just to find out that it wasn't as good as they one they had planned for. That is to say, Gorman's mostly actually wrong here. Now I think the pendulum often swings too far the other way, the T-72 was still pretty good especially in Soviet hands. And Gorman is allowed to be wrong because hes living through the fog of war, and anyway isn't quite trying to do what the title makes you think he is (classic Gorman). BUT I think that also makes this document, for our purposes, suspect. 

[dbsapp]

Sure, every assessment could be criticized. But at least it's something to rely on. 

I guess if CIA document of such kind stated the inferiority of Soviet tanks quality, it would be universally accepted on this forum as a final and ultimate proof

I would like to see the credible document that proves that t-72 or any late Cold war Soviet tanks had problems with daylight optics. Still, I've seen none.

[dbsapp]

52 minutes ago, BeondTheGrave said:

Take, for example, the fact that on pg. 12 he bases part of his conclusion on the fact that the T-72 had a laser rangefinder. As weve discussed, it didn't. It used coincidence, less quick and accurate.

It's pg.10, not pg.12.

And T-72 for sure had laser rangefinder. All t-72 family tanks have it, except for the very first t-72 of relatively limited production. All t-72 starting from t-72A were equipped with laser rangefinder.

By the time the doc was written the production of newer types were in full. 

 

Edited October 21, 2021 by dbsapp

[BeondTheGrave]

Youre right RE: The T-72, my bad. The -72A has the finder, the Ural does not. 

RE: the doc, what Im saying is that its not a CIA estimate. Its an Army complaint about inaccurate CIA estimates. Not only is this not a CIA piece, the majority of the statistics he provides (that is, the entirety of the second section) is lifted nearly verbatim from other TRADOC documents including the Yom Kippur Lessons Learned document. Dont get me wrong, its a super cool piece. I appreciate that you linked it, I was not familiar with it, but Im glad to be now. All I'm saying is that what were discussing here is very limited in scope that that piece doesn't really address it in the way we'd need it to to compare to our in game observations. IMO. 

If I had to guess, probably the best RL data well get is from the Gulf War and the lessons drawn there, but I think many here will agree that that data is itself skewed in a different direction. Its closest in timeframe to the CMCW timeline, its the closest in terms of equipment, and its a battlefield scenario rather than a gunnery test. 

[chuckdyke]

Here is what the Tank Museum has to say about the T72. Sure enough it can have an M60 for breakfast. 

T-72 main battle tank (1972) (tanks-encyclopedia.com)

[The_Capt]

11 hours ago, Centurian52 said:

One more time with the T80B. Surprisingly it did significantly worse than either the T64A or the M60A1. I suspect the issue is that 10 tests is too small of a sample size and it just had a run of bad luck. Or maybe it actually does have worse spotting.

T80B spotting M60 at 2k

(?) spot     firm spot   (in seconds)
32             39
207           221
59             80
99             106
252           259
75             82
115            115
84             91
173           173
94            102

average for (?) spot: 119 seconds

average for firm spot: 126.8 seconds

Ok, did a test run with the T80, again a platoon - going to use platoons from here on out, it is a more accurate representation of how the game is supposed to be played as it employs the Tac C2 system, it scales nicely as demonstrated (and it is a lot faster).  And so I think you are correct, looks like the T80 spots worse (about 10 seconds longer as a group):

So this is comparable tests when scaled together) about a 32% increase for me, and a 28.5% increase to spotting time from yours.  Not sure why this is happening, would have to unpack the difference in optics for the T80 vs T72.  But what is really interesting, and probably more important from an in-game perspective is the Mean Time to Hit.  The T72s took on average 56.2 seconds while the T80s are coming in at 44.5 seconds, so I would probably prefer taking my chances with the T72s to be honest.  The T80s take longer to see you but then less time to kill you.  

[akd]

15 hours ago, Centurian52 said:

One more time with the T80B. Surprisingly it did significantly worse than either the T64A or the M60A1. I suspect the issue is that 10 tests is too small of a sample size and it just had a run of bad luck. Or maybe it actually does have worse spotting.

T80B spotting M60 at 2k

(?) spot     firm spot   (in seconds)
32             39
207           221
59             80
99             106
252           259
75             82
115            115
84             91
173           173
94            102

average for (?) spot: 119 seconds

average for firm spot: 126.8 seconds

10 is too small, and you are not controlling for all tanks spotting the same object (and now tests are being introduced into the discussion with even fewer controls).  You cannot make any conclusions about relative spotting performance of an individual tank with these kind of tests, beyond simple can spot / cannot spot.

 

Conclusion I would take away form this test (and frankly all the others being posted here) is…there are some numbers.

Edited October 21, 2021 by akd

[The_Capt]

7 minutes ago, akd said:

Conclusion I would take away form this test (and frankly all the others being posted here) is…there are some numbers.

Don't agree with this entirely.  Nothing can be taken as definitive but we can see some trends, whether they carry through or are bumps on a much larger curve is an unknown.  Some of these trends appear consistent but we are still well into abductive reasoning here, which is fine we have gone to war in RL with less.

The hardest thing in not the numbers, it is the lack of RL data at this point,  We can say "CM is showing trends along the following behaviors..." but we cannot really tell if they reflect RL outcomes.  We could run 1000 tests sets and still not know how it links to reality.

All that aside, what I am seeing as vague outline trends (taken with significant conditions) make sense to me based on what we can glean from second and third hand anecdotes [Note: John Kettler already posted that CIA doc, and I gave my concerns about it on another thread].  

[Centurian52]

3 minutes ago, akd said:

10 is too small, and you are not controlling for all tanks spotting the same object.  You cannot make any conclusions about relative spotting performance with these kind of tests, beyond simple can spot / cannot spot.

 

Conclusion I would take away form this test (and frankly all the others being posted here) is…there are some numbers.

???? We are in agreement that a larger sample is needed to make any definitive conclusions. But what do you mean that I am "not controlling for all tanks spotting the same object"? There is only one object to spot (the M60 at 2k), and only one tank doing the spotting (the T80B). There are no other forces on either side. And apparently I got roughly the right figure anyway, since it seems to be in agreement with The_Capt's tests. Based on his earlier tests spotting time should be inversely proportional to the number of tanks in communication with each other. If that is true then if my lone T80 could spot a single target at 2k in an average of 126.8 seconds, then I would predict that a platoon of four T80s would spot that same target at that same distance in an average of 31.7 seconds. And The_Capt got an average of 31.4 seconds, which is pretty close to what I would have predicted.

The numbers by themselves mean very little. But as we run more tests we gain confidence that the numbers are accurate. And as we run different tests we see how those numbers differ in relation to earlier tests with different conditions. The first tests I did with the T64A meant very little besides "here are some numbers". The_Capt's tests demonstrated a linear relationship between spotting times and distance and the number of spotters within C2 communication. My second set of tests suggested similar spotting performance between the T64A and M60A1, and my latest set of tests suggest that the T80B has poorer spotting performance than the T64A and M60A1. These tests are revealing interesting information, even if it isn't perfectly precise of exhaustive yet.

I would like to do more exhaustive testing (100 trails for every variable changed, and try changing more variables such as trying the whole list of vehicles, trying at different distances, with different crew skill levels, buttoned up and opened up, with different lighting conditions, both single vehicles and platoons, in the open, in hull down, in trees, etc...). But I only have so much time.

[BeondTheGrave]

23 minutes ago, The_Capt said:

The hardest thing in not the numbers, it is the lack of RL data at this point,  We can say "CM is showing trends along the following behaviors..." but we cannot really tell if they reflect RL outcomes.  We could run 1000 tests sets and still not know how it links to reality.

Interestingly, a whole trove of Gulf War documents should be releasing sometime in the near future. And by near future, probably not within the lifetime of this product near future, but soonish. We just hit the 30yr classification mark this year. 

I am still convinced that Gulf War data is probably the best bet for the T-72. Other options might be from Eastern European countries right after the end of the Cold War. Anybody know what happened to East Germany's T-72s after unification? Id guess though that any technical data provided to the US by unified Germany would still be unavailable to the public.  

[dbsapp]

If it is agreed that we are lacking any RL data on actual performance than it immediately raises the question... what data is CMCW based upon? 

[The_Capt]

1 hour ago, dbsapp said:

If it is agreed that we are lacking any RL data on actual performance than it immediately raises the question... what data is CMCW based upon? 

I work inside the wire and even I don't know that and BFC would not/should not just be giving that sort of info out - especially not for free. [Aside: you keep asking that question, to the point I am getting suspicious of the motives...but I am old and paranoid]

I strongly suspect that they have built some sub-models under the hood that drive the engine.  For example, I am think there is a ballistics model that determines projectile versus armor based on things like round speed, diameter and density versus armor thickness, angle and quality.  

For things like spotting, I suspect that have taken a 360 horizon and broken it up on probability curves factoring a whole bunch of things, such as light, weather, soft human factors, target size/profile and behavior. 

Depending on the sophistication of the models the real-world data may to be too high resolution to be any use here.  For example the real world data for ballistics are not going to tell us anything without the ballistics model itself.  RL performance would be great if it looked like what holoween posted or how they came up with those graphs in that CIA-sponsored document.  I would be happy with examples of T72 engagements at 2km as a start.

The only thing we can add is that the model is good enough for UK Mod, so there is that.

Edited October 21, 2021 by The_Capt

[akd]

3 hours ago, Centurian52 said:

???? We are in agreement that a larger sample is needed to make any definitive conclusions. But what do you mean that I am "not controlling for all tanks spotting the same object"? There is only one object to spot (the M60 at 2k), and only one tank doing the spotting (the T80B).

Didn’t you also run a M60 test versus T-64, which had entered into the discussion for comparison, or am I thinking of another test? If not, then disregard.  You are correct that your T-64 vs. T80 appears to be a properly controlled test of relative spotting, just a low sample size that will be very prone to skewed averages because of the high variability (human factor) built into the spotting system.

@The_Capt, I agree that the small sample tests can at least indicate there is no huge problem, e.g. tank x can see in a situation tank y cannot, or tank x always spots before tank y.  But drawing conclusion that tank x is better at spotting than tank y based an 10 second (or more) differences in avg. spotting times from small sample sizes is faulty, even more so if you introduce more variables than 1 tank x spotting 1 test target.

Edited October 21, 2021 by akd

[Centurian52]

4 minutes ago, akd said:

Didn’t you also run a M60 test versus T-64, which had entered into the discussion for comparison, or am I thinking of another test?

Ah, I see what you mean. The type of object that the M60 was spotting was different (it was spotting a T64, rather than an M60, while the T64 and T80 were both spotting an M60). I assumed a tank sized target was a tank sized target, and simply playing the test scenario from the other side was the easiest way to keep all other variables identical (with the one caveat that I went back into the editor to change which crew was mounted and which was dismounted to avoid any shooting interfering with the results). But since the M60 is taller than T72oids that might not have been a valid assumption (that a tank sized target is a tank sized target). Perhaps I should try the test again with an M60 as the target for the M60.

[BeondTheGrave]

2 hours ago, The_Capt said:

I work inside the wire and even I don't know that and BFC would not/should not just be giving that sort of info out - especially not for free.

Call me a conspiracy theorist, but if CM were based solely on BFC's personal take on publicly sourced documents it seems to me that their tightlipped control on the even the basics of their model suggest that maybe theyre very paranoid that they'll get ripped off. That seems plausible, but IMO unreasonable as you cant really rip off the spotting mechanics, say, without ripping off the whole model. Alternatively, I just wonder if CM doesn't have some data in it drawn from confidential internal MoD information. I would wonder how long BFC has been working with the MoD and what the nature of that relationship is. Put it another way, perhaps they know a thing or two that goes beyond just archival research and mining documents. 

But if thats the case, that also means these kinds of conversations will never move beyond what we can glean from hours of gameplay and gunslinger tests conducted 10 rounds at a time. Nor will they ever admit "oh yeah its because the game is based on protected information."

Edited October 21, 2021 by BeondTheGrave

[dbsapp]

17 hours ago, The_Capt said:

[Aside: you keep asking that question, to the point I am getting suspicious of the motives...but I am old and paranoid]

And people call me conspiracy theorist!

[The_Capt]

1 hour ago, dbsapp said:

And people call me conspiracy theorist!

Well if this is corporate espionage there may be better ways to do it.

Seriously, BFC has invested 15 years in this engine and unlike us their livelihoods are at stake, things that families depend on, I get why they are going to keep things close to their chests.

We will just have to keep grasping at shadows…

[chuckdyke]

There are too many variables in the game to come to a sensible conclusion.