First test with Fireflies vs King Tiger

[Vanir Ausf B]

First, thanks for running tests.

Unfortunately, while your results are interesting, this amount of testing is not exhaustive or even very indicative. Assuming you're controlling for variables, I would like to see perhaps ten times this number of shots to really consider it indicative. That still wouldn't be definitive, but it would be enough for us to pursue it. There's a reason c3k spends a week setting up and running tests; a relatively small number of shots doesn't tell us enough.

Also, it would be good to know what the parameters of the your test are in more detail. Against stationary King Tigers, even a decently well-crewed Firefly with a straight shot should hit quite often at 500 meters. That's a big target, at fairly short range for a decent tank and crew.

I disagree the results are not indicating. In order for the APDS accuracy to conform to expected real world results future results will have to be dramatically different. That is very unlikely to happen.

As for other parameters, all crews are Regular and +0 leadership. The Firefly crews are normal motivation, the King Tigers are Fanatical. The KTs have short covered arcs to prevent them from returning fire (the fanatical motivation is to prevent them from disregarding the covered arc when they take fire). I'm not sure what other parameters are relevant.

I think I will move the range out a bit to see where we start to see first shot misses.

[StellarRat]

There aren't a lot opportunties to shoot 1000+ meter shots in European settings. In ROTC all the manuals said that tank combat would take place at 1000 or LESS even in the 80's (when I was in college) . I think a 500 meter test is probably more realistic. Now in Russia steppe 1000+ is very valid.

[Vanir Ausf B]

1. There are operational research documents that contradict the one you posted in this thread, with results showing equal accuracy for APCBC and APDS at certain ranges and results showing greater accuracy for APDS. However, I don't think any single one of these documents is the "right" one; this simply highlights that there is a complex issue of inconsistency at play here. This is discussed in depth in the thread you linked. In particular, read the very last post in that thread.

The very last post you mention accepts that APDS accuracy was inconsistent, but then suggests that modeling dud rate is not worth the effort. I take it that is also your position?

The takeaway I get from this is that further testing is waste of my time, since nothing is likely to be changed regardless of the result. It's fairly time intensive as each unit must be clicked on to note which type of ammo is fired for each shot. I'm willing to do it if the results will matter, but you seem to suggest they won't.

[Guest]

I disagree the results are not indicating. In order for the APDS accuracy to conform to expected real world results future results will have to be dramatically different. That is very unlikely to happen.

Except you literally cannot say this. I've just told you your sample size is not large enough to be significant. You can't just say "That's okay, because I think the next ten samples of this size will not vary dramatically from this one." That's the *opposite* of what you can say.

Note that I'm not disparaging your test. I'm telling you what we would need to see for that test to convince us there's a problem. We'd be looking at days of tester and developer time to look into this. We can't jump at every shadow or there would never be time to actually make the game.

As for other parameters, all crews are Regular and +0 leadership. The Firefly crews are normal motivation, the King Tigers are Fanatical. The KTs have short covered arcs to prevent them from returning fire (the fanatical motivation is to prevent them from disregarding the covered arc when they take fire). I'm not sure what other parameters are relevant.

Okay, good. So - the supposition as I understand it is that Fireflies with unlimited time to line up a shot, on what amounts to a firing range, at relatively short range against a target the size of a KT, will miss a significant amount of the time? Again, not disparaging: is this an accurate characterization of the parameters?

I think I will move the range out a bit to see where we start to see first shot misses.

Sounds good.

[Guest]

The very last post you mention accepts that APDS accuracy was inconsistent, but then suggests that modeling dud rate is not worth the effort. I take it that is also your position?

The takeaway I get from this is that further testing is waste of my time, since nothing is likely to be changed regardless of the result. It's fairly time intensive as each unit must be clicked on to note which type of ammo is fired for each shot. I'm willing to do it if the results will matter, but you seem to suggest they won't.

If your testing - or anyone else's - reveals an issue, we will look into it. OR reports are not ballistic gospel. Nor do opinions about what is important or worth modeling reflect what's in the code. If you feel the game is inaccurate, test, or get in touch with a beta tester who would like to.

[akd]

Phil, just a reminder, but there is no such thing as a "firing range conditions" in CMBN and as far as I know relative safety is not accounted for in time to aim, etc. I don't think that is a factor here, but the short range and size of the target certainly are important ones.

[hoolaman]

First, thanks for running tests.

Unfortunately, while your results are interesting, this amount of testing is not exhaustive or even very indicative. Assuming you're controlling for variables, I would like to see perhaps ten times this number of shots to really consider it indicative. That still wouldn't be definitive, but it would be enough for us to pursue it. There's a reason c3k spends a week setting up and running tests; a relatively small number of shots doesn't tell us enough.

Also, it would be good to know what the parameters of the your test are in more detail. Against stationary King Tigers, even a decently well-crewed Firefly with a straight shot should hit quite often at 500 meters. That's a big target, at fairly short range for a decent tank and crew.

Assuming the variables are being controlled, I would think 41 shots with a 100% success rate is a well on the way to being statistically significant, unlike the 10 shot sample that akd posted without any challenge.

In my slightly longer testing testing I saw stationary regular Sherman VC get hits on Tiger I's from every single one of about 20 shots in a row (not just firing APDS though). (At 850m range)

[akd]

Assuming the variables are being controlled, I would think 41 shots with a 100% success rate is a well on the way to being statistically significant, unlike the 10 shot sample that akd posted without any challenge.

In my slightly longer testing testing I saw stationary regular Sherman VC get hits on Tiger I's from every single one of about 20 shots in a row (not just firing APDS though).

FYI, I did not run that quick test as a valid sample, but just as a counter to your claim of 100% accuracy based on no test at all. If accuracy were anywhere near 100%, there is almost no way I could get those results in a 10-shot sample.

Now, however, you say you are running a controlled test. If so, I will be interested in the results if they are reproducible.

[sburke]

Except you literally cannot say this. I've just told you your sample size is not large enough to be significant. You can't just say "That's okay, because I think the next ten samples of this size will not vary dramatically from this one." That's the *opposite* of what you can say.

Note that I'm not disparaging your test. I'm telling you what we would need to see for that test to convince us there's a problem. We'd be looking at days of tester and developer time to look into this. We can't jump at every shadow or there would never be time to actually make the game.

I learned this lesson the hard way back when we were trying to verify the amount of protection or not a wall supplied. It wasn't until I started trying to create a really large testing environment which could generate a desired sampling rate that I was able to gauge the curve ... and ultimately stumbled onto a completely different issue.

I am sure Vanir this is probably making you want to tear your hair out, but really to engineer a computer program for this is a whole different deal than what you would face driving a tank up to a firing range. As noob found in his spotting tests, there are coding relationships that can skew your results with you being completely unaware they are there. I feel for you man, but there aren't any shortcuts.

[Vanir Ausf B]

Ok, I will grind out a larger sample size, if for no other reason than to prove my first results were not a fluke. I will try 700m and see how that goes. While I expect a drop in accuracy overall I will absolutely eat my shorts if any significant difference between AP and APDS develops.

Okay, good. So - the supposition as I understand it is that Fireflies with unlimited time to line up a shot, on what amounts to a firing range, at relatively short range against a target the size of a KT, will miss a significant amount of the time? Again, not disparaging: is this an accurate characterization of the parameters?

Correct. Keep a couple of things in mind. As AKD pointed out, the AI doesn't know it's on a gunnery range. Also, the real world tests referenced earlier were also conducted on a firing range, by people who knew they were on a firing range and at targets that the exact distance to was known.

[dieseltaylor]

Source: PRO document WO291/180, OR report on accuracy of anti-tank gunnery.

If the 17pdr is lined up and ranged it will hit a Tiger VI 100% of the time on a first shot all the way out to 1500 yards. At 500 yards if not ranged or not lined it drops to 98%. If you go out to 1000yards and adjustment required the first hit chance drops to 46%.

The 6pdr figures are 100%, 87% and at 1000 drops to 33% .

The probability of a second round hit is 100% at 500 yards for both, and at 1000 94% and 86% respectively.

The recommendations are

These values are those plotted on the graphs in the report; the value marked (e) is interpolated from

other data points. The overall shape of the fitted curves in each case is sinusoidal.

This report recommends that the maximum range of engagement for 6-pdr and 17-pdr ATk guns be considered 800 and 1000 yards respectively. The criteria stated for maximum range of engagement for a statically-sited ATk gun are:

50% chance of first-round hit on a static hull-up target;

90% chance of subsequent rounds hitting a static hull-up target;

50% hits on a hull-up direct-crossing target moving at 15 mph after MPI roughly corrected;

50% hits on a static hull-down target after MPI roughly corrected.

The first table clearly shows that errors in range have a much more important effect on accuracy than errors in line.

Penetration ranges against Pz VI for each gun (ammunition not specified) are stated as being 800 yards for 6-pdr on the front, 1600 yards on the side, and 2000–2500 yards for 17-pdr.

Hit probability is therefore regarded as a more important limitation on maximum engagement range than penetration.

Being ATG's making sure of a kill must be more important perhaps than for a tank who can move position!

[Vanir Ausf B]

I am sure Vanir this is probably making you want to tear your hair out, but really to engineer a computer program for this is a whole different deal than what you would face driving a tank up to a firing range. As noob found in his spotting tests, there are coding relationships that can skew your results with you being completely unaware they are there. I feel for you man, but there aren't any shortcuts.

I've been through this before. I was the guy who did the initial tests on cover behind walls, as well as many follow-up tests, and started that thread.

There is really no other way to test this than on a virtual firing range.

[dieseltaylor]

Probability (%) of hitting hull-down target with first round:

Gun -- 500 1000

17 pdr - 59 18

Probability (%) of hitting static hull-down target after first round:

Gun ----------500 1000 1500 2000 2500

6 pdr---------- 85 -43 ---22 ---14

17 pdr ---------88 -51 ---29 ---18 -12

[akd]

Ok, I will grind out a larger sample size, if for no other reason than to prove my first results were not a fluke. I will try 700m and see how that goes. While I expect a drop in accuracy overall I will absolutely eat my shorts if any significant difference between AP and APDS develops.

Why 700m?

at targets that the exact distance to was known.

Are you 100% certain on that? I have not seen documentation of WO 293/1263 that states if exact ranges were known by gunners. I've also seen it stated that these tests were against 5'x2' targets, not 6'x6' targets.

edit: if this is an accurate quotation, then range was known but target was also smaller than originally stated:

WO 291/1263, Firing Trials, 17pdr Sherman

"Table VI has been constructed which shows the probability of a hit on a target 5' wide by 2' high

(representing a Panther turret) at various ranges using both types of round."

Range (yards) APC % AP/DS %

400 90.5 56.6

600 73.0 34.2

800 57.3 21.9

1000 45.3 14.9

1500 25.4 7.1

Comments and corrections

These assume that the MPI is placed centrally on the target.

The trace from the AP/DS round was not seen in 73% of cases by a flank observer, and in no case from

inside the tank.

Note: This data is for a known range (no estimation error) and MPI already put on target. Its value is in that it shows that hitting a hull down target is difficult for this HV gun in particular. Throw in some range estimation error (and other smaller errors) and you have a problem.

[akd]

But that is really neither here nor there. I don't think the issue here is absolute accuracy of either round, but the relative accuracy of APDS vs. APCBC. This report says less accurate, other reports say the same or more accurate, so that is huge problem if we want to show anything by comparing reports to in-game tests, even if we setup the the parameters in-game to perfectly match this one particular test.

[Guest]

Phil, just a reminder, but there is no such thing as a "firing range conditions" in CMBN and as far as I know relative safety is not accounted for in time to aim, etc. I don't think that is a factor here, but the short range and size of the target certainly are important ones.

Firing range conditions: units shooting at a stationary target, with no incoming fire.

You would need to see the algorithms to know what is accounted for, and what is and isn't a factor.

Assuming the variables are being controlled, I would think 41 shots with a 100% success rate is a well on the way to being statistically significant, unlike the 10 shot sample that akd posted without any challenge.

In my slightly longer testing testing I saw stationary regular Sherman VC get hits on Tiger I's from every single one of about 20 shots in a row (not just firing APDS though). (At 850m range)

No. 41 shots with a 100% success rate is very possibly a fluke, or a repetition of the same shooting cycle (or set of shooting cycles) over and over again. If you're simply reloading the same save over and over, for instance, you're removing variables that help to create overall randomness. These aren't variables you want to control when you're looking at the results of an overall system.

Put a hundred identical firing tanks on a range with a hundred targets. Have them fire a volley. Count first round hits. Reload the scenario from scratch, and do it again. Do it as many times as you like (ten would be a good start). Halfway through that number of tests, restart the game. If you guys are still getting the results you *think* you're getting currently... then we're talking.

[Vanir Ausf B]

But that is really neither here nor there. I don't think the issue here is absolute accuracy of either round, but the relative accuracy of APDS vs. APCBC.

Exactly. That's what I've been trying to say.

As for the 700m range, that was IIRC the average tank engagement range in NW Europe. I may change it depending on the distribution of first shot ammo selection. At 500m the AI shoots APDS first just under 60% of the time. At 1000m it shoots AP first over 90% of the time which makes testing APDS first shot accuracy at that range prohibitive.

This report say less accurate, other reports say the same or more accurate, so that is huge problem if we want to show anything by comparing reports to in-game tests, even if we setup the the parameters in-game to perfectly match this one particular test.

As I said earlier, I am aware that there is a likelyhood that nothing will be changed regardless of any test results since almost any likely result can be rationalized.

It would be so much easier if someone would just ask Charles if APDS and AP has the same accuracy. I think it unlikely he is unaware of the APDS tests referenced earlier. The poster named "Rexford" whose quotes I was pulling from the old forum thread was a man named Lorrin Bird, whose work on slope effects were used in the CMx1 games' ballistic modeling. The book he later co-wrote, World War II Ballistics: Armor and Gunnery is one Charles almost certainly has a copy of.

[Vanir Ausf B]

Put a hundred identical firing tanks on a range with a hundred targets. Have them fire a volley. Count first round hits. Reload the scenario from scratch, and do it again. Do it as many times as you like (ten would be a good start). Halfway through that number of tests, restart the game. If you guys are still getting the results you *think* you're getting currently... then we're talking.

You win. I am not going to do that. Especially when the result I'm getting may well be intended behavior. Carry on. Everything is fine. :cool:

[Steiner14]

It should not be forgotten that all these numbers are referencing ONLY the accuracy of the gun: distance is exactly known and aiming is 100% accurate.

But on the field the distance must be estimated.

The flatter the trajectory of a gun, the higher the estimation error may be.

And after the range is estimated, then it must be aimed (optics come into play!).

For example the Tiger's 8,8 KwK36 IIRC was shooting SPOT ("Fleck") up to 1000m (1100 yards).

This means an accuracy of 100%!

But additionaly to the extremely high accurcy it also had a very flat trajectory compared to other guns. What does that mean? It means, that the flat trajectory allows a wider estimation error, and therefore a wider range of chosen sights, while agun with a stronger curved trajectory, would already have no chance to hit with the same sight.

When it comes to hit small targets, i.e. dug in ATGs or hulldown tanks/TDs, the importance of a high precision gun becomes even bigger: if the range is correct and aimed correctly, then it will hit. Not so with the less accurate gun: it still has a high probability to miss.

And as third important point comes the much better optics into play: it increases the capability that aiming will be precise.

That makes three additional factors adding on each other in favor of the german guns. I think this really needs further investigation.

Another question is, if BFC really wants to model it accurately...

[JonS]

the Tiger's 8,8 KwK36 ... an accuracy of 100%!

You know, Steiner, if you told me the time I'd have to ask for a second opinion before believing it.

Assuming I could be bothered. Which I most likely couldn't.

[akd]

Firing range conditions: units shooting at a stationary target, with no incoming fire.

Sorry Phil, I misunderstood what you were saying there.

[Steiner14]

The discussion about range estimation as one decisive factor in gunfights, brings into focus that defending ATGs/tanks/TDs already have a priori knowledge about the distance of their targets. Resulting in reduced range estimation errors.

An idea to model this: cover arcs working as TRA (target reference area) for guns/tanks/TDs on the defending side (in case of attack/defend scenarios). As long as the unit is not moved a certain distance, in the shooting equations the error of range estimation could be reduced.

[akd]

The discussion about range estimation as one decisive factor in gunfights, brings into focus that defending ATGs/tanks/TDs already have a priori knowledge about the distance of their targets. Resulting in reduced range estimation errors.

An idea to model this: cover arcs working as TRA (target reference area) for guns/tanks/TDs on the defending side (in case of attack/defend scenarios). As long as the unit is not moved a certain distance, in the shooting equations the error of range estimation could be reduced.

TRPs work as "TRAs" (radius = 50m, I think) for friendly guns.

[sburke]

TRPs work as "TRAs" (radius = 50m, I think) for friendly guns.

IIRC this was how you emulate bore sighting an AT gun from way back in CMBO days.

[Steiner14]

TRPs work as "TRAs" (radius = 50m, I think) for friendly guns.

TRPs are needed for artillery and are much to scarce to model the advantage of known ranges for the defenders guns.