Hull down test (PzIV H, M4A1)

[HistoryLover]

Altipueri,

I'm not paid by Battlefront, it is me who pays them as customer. I believe I have presented enough facts that they should look at it. But so far I don't even know, if they looked at this thread. All I have is a feedback from a betatester who demanded a certain test which I did and since then nothing was heard again from him.

Btw, the testfile I provided was also not checked by them yet (0 downloads).

I will definately not invest more time.

[Mord]

C'mon man...you're gettin' kinda hyperbolic. ck3 didn't "demand" anything, he gave you some suggestions that would help further the cause. They are in the midst of making two games so that might be why they can't drop everything and post in this thread. Just be patient, things don't happen here over night...but they do happen—MG tweaking is proof among many other things.

Mord.

[sburke]

heh heh yeah never take no response as meaning anything more than that you just haven't heard anything. altipueri was just making a joke. BF will rarely look at anything without data, however having provided data does not mean you are going to get any kind of immediate reply. Even assuming that they absolutely agree with you, changing the behavior of units has to be looked at closely so that they just don't break something else. It isn't just like they need to change the font.

[HistoryLover]

I hope you are correct.

[c3k]

Here is my response: your tests are worthless in my view.

Is that harsh? Yes. But let me explain, and then see if your anger is righteous.

You are testing HIT/KILL chances. That may be of interest to you, but not to me. Nor, in my opinion, does lethality matter for purposes of Hull Down spotting. I have no doubt that a larger target (tank in the open) is easier to hit than a smaller target (tank hull down). Let me present an analogy. It is easier to me to hit a 12 foot circle at 100 yards on the firing range with a scoped rifle than it is to hit a 3 foot circle at 100 yards. (The 12 foot diameter target is SIXTEEN times greater in area than the 3 foot circle.) Yet, I would be incompetent if I missed even 1 shot into the 3 foot circle with a scoped rifle from 100 yards. Target size, in this case, has NOTHING to do with chance of hit/chance of kill. Neither does your test.

What is curious to me, as a beta tester, is the SPOTTING TIMES. Will a hull down target be spotted before an exposed target would be spotted at the same range, by the same observer? If so, at what range, if any, does it crossover. Testing hulldown AGAINST exposed does not test spotting time. There needs to be minimal variation on the tests. (Hulldown tank shooting at exposed tank is no test for exposure or spotting.)

You've done a lot of work. It -hints- at an issue, but your test setup (from what I understand) simply doesn't test what needs to be looked at. Eventually, I will probably get around to testing this. My test will have tank X, hulldown, being spotted by tank A. The next test will have tank X, exposed, being spotted by tank A. Both tests will be at the same range. Then I will start to change the range. Then I will change the button up status. In all cases, tank A will have its hull facing AWAY from tank X, and the turret STARTING with a covered arc towards (but not including) tank X. Tank X will have a short covered arc. No one will shoot a single shot. It will have about 1,000 iterations. Or more.

The test will be one to assess the TIME to spot.

Will I post the results here? It depends.

So, whereas I -appreciate- your effort, I do not see any -value- in it as far as determining any hull down spotting errors.

But that's all my opinion. My record for these things is good, but not great. I make a lot of mistakes, false starts, dead ends and errors. Eventually, if there is a game error, it can be found. Consistency and elimination of variables is the only way it will be convincingly presented to BFC.

Consider your thread, again, to've been noticed by a beta tester, and hence BFC's testing process.

Regards,

Ken

[HistoryLover]

I have posted the numbers of spotted tanks in the first minute.

Maybe you don't know the importance of spotting first in the game, but I can assure you, it is of utmost importance. If you can't see already with these numbers, that the model doesn't work, then I'm sorry.

But my "worthless" test showed even more: that the used model obviously is so extremely flawed, that it also does not reflect the hit probability depending on the SIZE of the target in - at least - a plausible range.

Btw, how comes that you super-tester in all the years have not recognized that tank duels are like throwing dices? After these discoveries, if I were a beta tester, I would be less arrogant and very humble.

[pnzrldr]

Ken,

Yeah, harsh man. While hit/kill numbers are not an exact data set for a programmer looking at this, given what we know about the specific tank performance with other variables removed as much as possible (crew quality, ammo loadout, weather, etc...) the hit/kill sequence should logically follow the spotting by a bounded period of time (hit/kill follows spot by X seconds, plus/minus Y, in a presumably standard deviation). So while not a solid data point, they still point to a potential problem.

For what it is worth, speaking as someone with long hours spent scanning as a TC, I concur that the number of eyes is the key variable. But some eyes are more valuable than others. The TC spends virtually ALL of his time scanning. He also knows the tactical situation to a degree and can therefore prioritize where he is looking, spending more time on more 'likely' locations. I usually used a set of binos as well, which are a significant help. A smart TC always establishes a logical sector for his gunner to scan. Usually, the TC will set right/left limits for a horizontal scan by the gunner, based upon likely enemy locations and how far out the scan is looking. Obviously the usual assumption is that the gunner should look long range (w/ ten power optics) and the TC should focus on things closer in that are easily identified w/ naked eye. In this configuration, the TC still usually finds the targets as it is easier for the naked eye to key in on movement, which is what usually allows you to spot something. For stationary targets on the offense... well, the M1 I used cheats of course, since it has thermals and a superb stab system. But even when moving that is no sub for the TC staying up and scanning. At gunnery, we typically have an engagement with the stab disabled. This is HARD. The gunner really cannot scan while the tank is moving and there is no stab system. However, the loader usually comes up if you are moving and not shooting, and he lends additional eyeballs to the TC. If you are buttoned up w/ no stab and moving, I cannot imagine anyone ever seeing anything w/o advanced optics - like thermals and CITV. Like, nothing. Ever.

The driver did occasionally spot things first though, even seeing the world through his narrow little POV. In an ideal world the crew positions would be weighted, with the two having optics (TC [binos] and gunner) getting 3 or 4 times the weight assigned to loader, and 5 times that assigned to driver. Not sure the spotting model is this advanced though.

History Lover, you have a plethora of betas looking at this thread and a new post going on the beta forum. Despair not. Remember that confirming that you can see a problem in the way the game behaves is radically different from providing relevant data that a computer programmer can use to change the simulation. The betas aren't listened to all that often - we have to make sacrifices to a strange demi-God named Charles - who likes South African wine apparently - to get any attention. But sometimes our entreaties and your prayers are heard. Keep the faith, keep asking hard questions, listen to the feedback and criticism, and set aside some money for the Eastern Front CM coming out in the near future. I promise you it will rock.

[Vanir Ausf B]

I don't see much point in more testing of hull down status. From my testing in the other thread I think we have a good basic idea of how hull down affects both spotting and hit probability in the game as well as the practical implications. There is an endless list of variables you could jigger this way or that, different vehicles with different optics, buttoned/unbuttoned, range, crew experience level, ect. and if someone wants to see how those affect things that's great. Logically, I would not expect any of those to fundamentally change the dynamic as I have presented it, but bugs are often found by not assuming everything is working logically.

1000 iterations per test is far more than necessary, IMO. I did 63 per side initially, which is what I posted. I went back last night and did 45 more per side. After adding in the new results the averages hardly budged, going from 84.5 to 83.2 for the hull down tanks and from 57.4 to 57.7 for the tanks in the open.

I will do a few more spotting tests tonight regarding how optics and vehicle size affects spotting.

[altipueri]

heh heh yeah never take no response as meaning anything more than that you just haven't heard anything. altipueri was just making a joke. BF will rarely look at anything without data, however having provided data does not mean you are going to get any kind of immediate reply. Even assuming that they absolutely agree with you, changing the behavior of units has to be looked at closely so that they just don't break something else. It isn't just like they need to change the font.

Blimey, I didn't know I had made a joke. I apologise.

I used to do this stats stuff in the days before spreadsheets and electronic calculators. But I'm too old and crusty now.

After I made that post I played racketball and in the bar discussion after I was talking with an actuary friend who really does understand this stuff. I raised this very issue, two series of about sixty numbers, can you tell whether they come from a normal distribution (i.e distributed about a mean (=average) and 95% of results will fall within 2 standard deviations of the mean. Initially I just gave the figures, without saying what they represented, only later saying they notionally represented time in seconds to spot a tank at x yards. That sequence of results does look like a "normal distribution". (This sort of stuff does not affect most of your lives until you try to understand why your pension is so small.)

Quick rule of thumb test - about 30% of a random set of numbers will start with a "1" (I don't remember the full explanation, but it was discovered by a monk in about 1800 who noticed that certain pages of his logarithm tables were more worn than others.)

Conclusion: there is a significant enough difference in those two sets of numbers to warrant further investigation.

PS - I live in Cheltenham, UK home to GCHQ (=NSA) "Spies R 'Us"; the place is crawling with spies, spooks, mathematicians, computer nerds, linguists and various other forms of low life, many of whom can carry on conversations in four languages at the same while doing a crossword and playing three dimensional chess. It's depressing, but good if you need someone to join a quiz team.

[sburke]

Btw, how comes that you super-testers in all the years have not recognized that tank duels are like throwing dices? After these discoveries, if I were a beta tester, I would be less arrogant and very humble.

Yep that's us, all totally incompetent. Thank god you came along to point out the obvious...oh wait you didn't, Kulik did.

Thanks Kulik!

[Vanir Ausf B]

Ken,

The driver did occasionally spot things first though, even seeing the world through his narrow little POV. In an ideal world the crew positions would be weighted, with the two having optics (TC [binos] and gunner) getting 3 or 4 times the weight assigned to loader, and 5 times that assigned to driver. Not sure the spotting model is this advanced though.

Test results suggest it is not advanced enough.

The problem can be reduced to the difference in two variables: spotting ability reduction caused by LOS obstruction while hull down vs. concealment bonus for hull down tanks. The former is much larger than the latter, about 67% vs. 16% according to my test results (there appears to be no concealment bonus at all for being partially hull down). I will let others with personal experience in the matter debate exactly where those numbers should be, but they need to be much closer together. Do that and the problem is solved.

There does not appear to be any issue with hit probability related to hull down. My testing has been fairly limited on this, but it shows a reduction of first shot hit percentage of more than 50% on hull down or even mostly hull down tanks at 800 meters.

[c3k]

Arrogant? Hardly. I'm sorry if my opinion seemed harsh. A lot of the tabular data presented doesn't make much sense. You wanted to know why you hadn't heard anything. I told you. You've gone off into a testing regime I deem worthless. As I alluded to, I've been wrong before. I may be wrong now. However, you've ignored my "advice". Shrug. You're free to do so. I'm free to ignore your results. I am doing so.

Just for grins, I'll tell you of an early test I did on spotting. I ran 100 dismounted tank crews through a sniper gauntlet. Those which ONLY had a Driver and a TC left, were safely let through back to their tanks. THAT pre-condition took a LOT of work. That was to remove the effect of the gunner and his optics from the spotting routine. I drove the few tanks into a hull down position to blind the driver. I did this a year and a half ago, spending over 2 weeks on it. In the end, I found nothing. I do not lightly waste my time.

Regards,

Ken

[Vanir Ausf B]

But my "worthless" test showed even more: that the used model obviously is so extremely flawed, that it also does not reflect the hit probability depending on the SIZE of the target in - at least - a plausible range.

I don't think your tests are worthless. But there are a few issues with them.

1) The whole turning the turret backwards thing. I know that was suggested to you, but there are less convoluted ways of determining the effects of removing the driver and radio operator from the spotting equation. And I've already done that anyways.

2) Your sample sizes are rather small.

3) Using tank damage as a means of measuring hull down effectiveness is problematic since you are introducing a factor that is not directly related to hull down status. It is better to factor out variability in damage per hit by assuming every hit destroys the tank and then only counting the first shot fired.

As someone who has done this sort of thing a time or two before I can tell you that it is unusual for everyone to simultaneously nod their heads in agreement with you, even when you have a strong case. Good data and persistence will usually win out in the end. But sometimes you just have to accept that BFC's priorities are not your priorities (see recent Panther shot trap thread for an example). And sometimes the wheels of change turn at a glacial pace. The issue of AT rockets fired from buildings was hotly debated for close to 2 years before any change was made. Again, persistence is key. You'll never get anywhere if you throw your hands up every time a beta tester poops on your spreadsheet.

[Battlefront.com]

Sigh. If I had a dime for every time a customer made a flawed test, made flawed conclusions, then got pissed off at everybody and anybody who tries to help make it better... I'd not have to read threads like this because I'd be a rich man

I have posted the numbers of spotted tanks in the first minute.

Maybe you don't know the importance of spotting first in the game, but I can assure you, it is of utmost importance. If you can't see already with these numbers, that the model doesn't work, then I'm sorry.

A test without rigorous control of variables to test a very specific theory is usually not worth much. Even if repeated thousands of times instead of just a few times.

But my "worthless" test showed even more: that the used model obviously is so extremely flawed, that it also does not reflect the hit probability depending on the SIZE of the target in - at least - a plausible range.

Then your test is wrong, because it does reflect the hit probability depending on size and a ton more. It's all physics, no dice rolls.

Btw, how comes that you super-tester in all the years have not recognized that tank duels are like throwing dices?

Because it doesn't work that way so why should they think it does?

After these discoveries, if I were a beta tester, I would be less arrogant and very humble.

"Discoveries"? More like wild and unfounded theories to me.

Talk about arrogance... you're one person making one test and you suddenly know more about how the game works than people who have been playing and/or testing since the game was made. Not to mention knowing more than the people who actually made the game itself.

Bottom line here is there could very well be something that needs to be improved, but based on your test I don't see what that is.

Steve

[Battlefront.com]

I don't think your tests are worthless. But there are a few issues with them.

From my position this is nonsensical. A test that isn't rigorously designed is by definition worthless in and of itself. At best it might produce a result that is curious to pursue making a better test. However, good tests are difficult to setup and in 15 years of doing this I've found that few are up to the challenge. Even if it is purely due to a lack of time (as C3K indicated, sometimes these test are MASSIVELY time consuming).

1) The whole turning the turret backwards thing. I know that was suggested to you, but there are less convoluted ways of determining the effects of removing the driver and radio operator from the spotting equation. And I've already done that anyways.

Often there's more than one way to eliminate a variable. But it is important to know that it needs to be eliminated when designing a test.

2) Your sample sizes are rather small.

This is the #1 flaw in tests seen over the years. It's also the most time consuming part of tabulating data, so it's not surprising people try to short cut.

3) Using tank damage as a means of measuring hull down effectiveness is problematic

You're being overly kind here. It actually has absolutely nothing to do with hull down at all, in the least. Easily proven too. Put a 37mm AT Gun up against a King Tiger or Jagdpanther in hull down position. You can fire a million shells and probably only a handful would cause any damage. Or put a Pak43 up against a Stuart and see the exact opposite result. So if one is trying to determine effectiveness of Hull Down, obviously damage is absolutely not relevant. Any test that is based on something so fundamentally off the mark holds no value in terms of proving/disproving the premise.

gain, persistence is key. You'll never get anywhere if you throw your hands up every time a beta tester poops on your spreadsheet.

Precisely Beta testers are people that generally have had the crap kicked out of their testing in public, learned from their mistakes, and kept their chin up in the process. Insult chucking has never been a quality we've found applicable to the testing process.

Steve

[Vanir Ausf B]

You're being overly kind here.

Yes, I know But I have some sympathy for his position since he is not entirely incorrect, even if his tests are insufficient to prove it. As far as I can ascertain he has made basically 3 assertions:

1) The Panzer IV is at a significant spotting disadvantage vs. the Sherman.

I am in the process of testing this. Early results suggest it is probably not true.

2) Hull down tanks are no more difficult to hit than tanks in the open.

This is not even remotely true.

3) Hull down tanks are at a significant spotting disadvantage vs. tanks in the open.

This actually is true. As you have probably noticed, there is a whole separate thread on this although it seems to have spilled over into this one.

[altipueri]

For those who want to continue doing tests you may take some comfort from the fact that you don't need to do many before you can draw some meaningful conclusions. At least thirty should be enough for most tests, hence Vanirs 60 or so was a good choice. 600 would not give significantly more useful information.

Remember to change only one variable at a time.

You will of course eventually go mad.

[Vanir Ausf B]

You will of course eventually go mad.

If I were sane I would have never started.

It's been twenty years :eek: since I took statistics and I have inconveniently forgotten how to calculate confidence intervals. So now I run tests until the percentages stop moving around. Then I run a few more, post them on the forum and dare anyone to prove me wrong

[c3k]

3) Hull down tanks are at a significant spotting disadvantage vs. tanks in the open.

This actually is true. As you have probably noticed, there is a whole separate thread on this although it seems to have spilled over into this one.

...and this is the heart of the matter.

[BadgerDog]

1) The Panzer IV is at a significant spotting disadvantage vs. the Sherman.

I am in the process of testing this. Early results suggest it is probably not true.

I'd be really curious about the results of this testing ...

I'm currently in a large CMBN match with nothing but PzIV's against Shermans. The map is huge and the engagement distances are very long range.

I'm getting the floor wiped by Shermans who spot my PzIV's, even sitting in wood lines and I have yet to see one of them first. In fact, I'm lost 2-3 PzIV's to one Sherman that was actually sitting out in the open picking them off. It wasn't until 2 turns later until a Jagd crew elsewhere on the map spotted it. I realize it's an anecdotal observation, but it's consistent in this particular match.

I thought the German tanks were supposed to have superior optics that shined at long ranges for spotting and accuracy... :confused:

Regards,

Doug

[HistoryLover]

You're being overly kind here. It actually has absolutely nothing to do with hull down at all, in the least.

Ahem, the bogged tanks are not the hull down ones.

The test is very conservative, because it gives the tanks in the open this disadvantage (hopefully no one expects a tank with damaged motor and tracks to perform any better than the undamaged version) AND with the additional disadvantage that they need to turn the turret.

Therefore the hd tanks in my test are at a clear advantage and nevertheless they are slaughtered.

I'm surprised that you don't see, that this test is the proove that at least the PzIV hd model delivers obvious wrong results.

My tests were to proove that the results are NOT plausible to say the least. It should be the testers that recongize such things, shouldn't they? This is not a false grenade count, this is the core of tank combat modelling.

I find it strange that customers are attacked for not spoon feeding the errors instead of being thanked for their input on things the testers and developers have not recognized.

Maybe a word on statistics and probabilities and that my samples were too few.

If the statistics model needs 100, or even only 50 iterations of TANK duels to get a meaningful representation in the probability curve, then IMO it would be already extremely flawed. Also from a gameplay perspective, because no player wants to lose three battles in a row due to 3-sigma events and then to win three in a row because the distribution hits at the other extreme.

My examples indicate, that it's very easy possible to lose a company of perfectly hd PzIV against a company in the open.

As engineer I don't need any more tests to see that this is extremely flawed.

Tanks are a scarce resource (also for the player) and therefore it is important that the probability distribution is quite narrow and based on hard factors.

If the game would need 50 or 100 iterations of tank duels to get any meaningful representation, then IMO the probability distribution would be waaay too wide and the random factors would outweigh the hard factors.

Instead of attacking customers not being able to spoon feed the developers, the discussion maybe should focus on which results deem plausible in such a situation?

Instead of excusing all results with a quite universal excuse, that the player just needs to play 20 battles to see OVERALL how realistic everything is and that everything was fine.

The limits of simulations are always determined by the expected real world results. The results determine if the model works, not the model itself.

Without any hard numbers from the developer the model must deliver, it's also not possible for the developer to set the influence of the random factors and therefore the probability curve accordingly.

Do not attack the messenger.

[Battlefront.com]

I'm surprised that you don't see, that this test is the proove that at least the PzIV hd model delivers obvious wrong results.

Your presentation is a muddle of confusing information and definitely incorrect assertions. People have tried to help you, you've been abusive. Which...

I find it strange that customers are attacked for not spoon feeding the errors instead of being thanked for their input on things the testers and developers have not recognized.

BS. It is exactly the opposite. We thrive on outside input and have made hundreds, if not thousands, of tweaks to our games over the years because of thoughtfully presented positions made by joe-average customer. And they are REGULARLY thanked. But you're being disrespectful towards us, abusive towards others, dismissive of contrary information, and generally coming off as someone with a bad attitude. So no, you're not going to find the testers or I fawning over your efforts. Because you're the one who is making this into some sort of battle, not us.

Instead of attacking customers not being able to spoon feed the developers, the discussion maybe should focus on which results deem plausible in such a situation?

If you had ANY idea how many times customers claim there is something wrong through bad testing you'd understand why we don't ask "how high" when someone with a tiny post count says "jump".

Instead of excusing all results with a quite universal excuse, that the player just needs to play 20 battles to see OVERALL how realistic everything is and that everything was fine.

You obviously have either not read much of what has been said here or don't understand it. Either way, it's coloring your judgement of what is really going on.

The limits of simulations are always determined by the expected real world results. The results determine if the model works, not the model itself.

Without any hard numbers from the developer the model must deliver, it's also not possible for the developer to set the influence of the random factors and therefore the probability curve accordingly.

True, but this is why it is so CRITICALLY important to identify specifically which factors are involved and to what degree they may be off the mark. And that's why your insistence that we should sit up and take notice of a test that has obvious flaws in it is not helpful.

Do not attack the messenger.

When the messenger attacks the one he's giving the message to, what do you expect should happen?

Steve

[HistoryLover]

Good luck with that attitude to play the teacher to customers and to demand from them begging for improvements as bootlickers.

To say it more clearly: the core model of a core aspect of this game is probably flawed! If this is not severe enough to come down from the high horse, then I don't know what else must happen.

I choose the language for the kind of ignorance I think is necessary, and I don't need YOU or any other self proclaimed teacher here to tell me what he believes is adequate or not. Was this clear enough?

[Rocky Balboa]

I don't need YOU or any other self proclaimed teacher here to tell me what he believes is adequate or not. Was this clear enough?

Unfortunately the ones you are trying to convince are the ones who must determine what is adequate or not. Your testing so far is at least adequate enough to get some attention and you've already been told that it is being scrutinized and evaluated.

Thousands of players and countless hours have been spent with this game and If the simulation was as flawed as you proclaim it to be then this would have come up before now.

I’m not saying that there might be something that needs to be adjusted but you're really taking this much too personally. I assure you no one has more invested in the success of this game than the developers.

I applaud and appreciate your contribution in trying to make this a better game but try not to throw the baby out with the bath water.

[mjkerner]

Aboorsayswhat?