Jump to content

AI cheats! (with real data)


Recommended Posts

To both Maastrictian and L.Tankersley,

I would urge you to reconsider your opinion that the AI is getting any help what-so-ever, via a "bug" or a cheat. I believe it is definitely NOT getting help.

During the computer thinking phase, it is plotting targetting orders. Thus we see the AI fire first IF we leave OUR target choice to the TacAI. You see, the TacAI isn't choosing the target for the AI's side, the AI's "what do I do this turn" AI is doing that.

That is why, when we give a manual targeting order, the discrepancy goes away. Or when we do a hotseat game, since we can leave it all up to the TacAI.

Link to comment
Share on other sites

  • Replies 250
  • Created
  • Last Reply

Top Posters In This Topic

Chief Wiggum says, "whoa whoa, slow down egghead!"

So let me get this straight...the tac AI on the Strat AI player side is getting a statistically significant higher kill ratio, when pitted against the tac AI on the Human player side.

So can anyone translate this into my expected extra losses due to the AI advantage in, say, the "Whitmann in the East" kursk tank-brawl scenario? That would be my idea of 'significant' information.

Ren

Link to comment
Share on other sites

Originally posted by Renaud:

Chief Wiggum says, "whoa whoa, slow down egghead!"

So let me get this straight...the tac AI on the Strat AI player side is getting a statistically significant higher kill ratio, when pitted against the tac AI on the Human player side.

So can anyone translate this into my expected extra losses due to the AI advantage in, say, the "Whitmann in the East" kursk tank-brawl scenario? That would be my idea of 'significant' information.

Ren

Heh, actually Ren, I'm arguing that it is not and that all the results merely point out a flaw in the way the tests have been conducted. If you look at L.Tankersley's test, his confirms my point if you realize that the "what do I do this turn AI" is not just sitting around idly but is plotting the target order while we (the human) said "eh, let the TacAI take care of it".
Link to comment
Share on other sites

Originally posted by Cameroon:

However, I did notice something interesting. When I allowed the Soviet TacAI to choose the target (as I did in all the tests), it took them about 5 seconds to acquire in clear terrain over 742 meters. The German side, however, when I let the TacAI target, took about 10 seconds to acquire.

Those were consistent. The Soviet's TacAI would acquire the German tank before the German's first shot. However, the German TacAI would not acquire the Soviet tank until after the first shot by the Soviets and sometimes not until after the second shot.

Perhaps this is intended to show hesitance to fire on a friendly vehicle? Would be interesting to see if the discrepancy is present and reversed in a T-34 v. T-34 matchup.

Edit:

From L. Tankersley's test with T-34s:

Trial 1 (human as Axis, no manual orders)

At T=0, all AI tanks were immediately targetting human tanks (red targetting lines). At T=2, 3 human tanks were targetting. At T=4, 7 human tanks were targetting. At T=5, 3 of the AI tanks had fired. At T=6, all ten AI tanks had fired, and only one human-controlled tank had fired (all were targetting).

Trial 2 (human as Axis, manual orders)

From the start, all tanks were targetting. At T=4, 4 of the human-controlled tanks had fired. At T=5, all ten human tanks had fired, as had 4 AI tanks. At T=6, all tanks had fired.

Trial 3 (human as Allied, no manual orders)

At T=2, 3 human tanks were targetting. At T=4, 9 human tanks were targetting, and 4 AI tanks had fired. At T=5, all ten AI tanks had fired. Human controlled tanks didn't begin firing until T=7, and some fired as late as T=10 (a few were knocked out before firing).

Trial 4 (human as Allied, manual orders)

At T=4, five AI tanks fired. At T=5, all ten AI tanks had fired, and 2 human tanks had fired. At T=7, all human tanks had fired.

Uh-oh, here's something new to test...

[and re-edited because I can't read page numbers]

[ October 30, 2002, 05:00 PM: Message edited by: Offwhite ]

Link to comment
Share on other sites

The AI does not CHEAT or get any bonus's over manually targeted units or ones it chooses to engage.

From a background in testing CM for almost three years now I can say that any test with less than 100 run throughs is not a good enough sample.

Having said that, and since I find STATS insanely boring I will have Charles peak in here and have a look around when he gets some time. Perhaps he can shed some of Brainac knowledge of his to help lift the fog on this matter.

Madmatt

[ October 30, 2002, 03:46 PM: Message edited by: Madmatt ]

Link to comment
Share on other sites

Originally posted by Cameroon:

That is why, when we give a manual targeting order, the discrepancy goes away. Or when we do a hotseat game, since we can leave it all up to the TacAI.

It should be noted that Warren's initial numbers that shows a discrepency are *with* a targeting order for the human player.

I encourage you to download my scenario, run some tests with it using targeting orders and see what you get. I've spent an hour today on this issue, I don't have any more time to devote. But if you think we've forgotten something when doing our tests then do them yourself and tell us what you get!

--Chris

Link to comment
Share on other sites

Originally posted by Offwhite:

</font><blockquote>quote:</font><hr />Originally posted by Cameroon:

However, I did notice something interesting. When I allowed the Soviet TacAI to choose the target (as I did in all the tests), it took them about 5 seconds to acquire in clear terrain over 742 meters. The German side, however, when I let the TacAI target, took about 10 seconds to acquire.

Those were consistent. The Soviet's TacAI would acquire the German tank before the German's first shot. However, the German TacAI would not acquire the Soviet tank until after the first shot by the Soviets and sometimes not until after the second shot.

Perhaps this is intended to show hesitance to fire on a friendly vehicle? Would be interesting to see if the discrepancy is present and reversed in a T-34 v. T-34 matchup.

Edit:

From L. Tankersley's p. 4 test with T-34s:

Trial 1 (human as Axis, no manual orders)

At T=0, all AI tanks were immediately targetting human tanks (red targetting lines). At T=2, 3 human tanks were targetting. At T=4, 7 human tanks were targetting. At T=5, 3 of the AI tanks had fired. At T=6, all ten AI tanks had fired, and only one human-controlled tank had fired (all were targetting).

Trial 2 (human as Axis, manual orders)

From the start, all tanks were targetting. At T=4, 4 of the human-controlled tanks had fired. At T=5, all ten human tanks had fired, as had 4 AI tanks. At T=6, all tanks had fired.

Trial 3 (human as Allied, no manual orders)

At T=2, 3 human tanks were targetting. At T=4, 9 human tanks were targetting, and 4 AI tanks had fired. At T=5, all ten AI tanks had fired. Human controlled tanks didn't begin firing until T=7, and some fired as late as T=10 (a few were knocked out before firing).

Trial 4 (human as Allied, manual orders)

At T=4, five AI tanks fired. At T=5, all ten AI tanks had fired, and 2 human tanks had fired. At T=7, all human tanks had fired.

Uh-oh, here's something new to test...</font>
Link to comment
Share on other sites

Originally posted by Cameroon:

To both Maastrictian and L.Tankersley,

I would urge you to reconsider your opinion that the AI is getting any help what-so-ever, via a "bug" or a cheat. I believe it is definitely NOT getting help.

I'm not expressing an opinion, as yet. I agree there aren't sufficient trials here to really state anything definitively. (I would definitely agree that if you don't issue a manual targetting order in the orders phase, while the computer player does, this gives the computer player an advantage. I don't see this as a problem.)

Regardless, I don't believe the computer player is getting deliberate help. It might be that there is an unintented interaction, or "bug" that might be giving the computer player an edge in some situations. Or it all might be chance variation.

Link to comment
Share on other sites

I also, am quite studiously not claiming anything smile.gif In fact, based on the 176 test I've run I'd say any difference is pretty small, if present at all.

Warren -- try right clicking on the link I gave and choosing "save as". That works for me at least.

Cameroon -- You are welcome to do whatever tests you want and post the results, but I think we wil all get farther if we all concentrate on using the same testing scenario, varrying only by the targeting orders we give. That way we can combine all our results and see what we get, rather than having 20 different tests testing different things none of which is significant by itself.

I strongly agree with Madmatt's comment that we need more than 100 tests to get anywhere, preferably, in my mind, more than 1000. I can perform that many, but it will take me a week. It will be much quicker if everyone simply runs my scenario once or twice.

(note, I'm not at all hung up about whose scenario gets run. Its just that I'm the only one whose posted a scenario for others to run at this point.)

--Chris

Link to comment
Share on other sites

Originally posted by Cameroon:

One flaw that I see in the majority of these test scenarios (I commented on it above, but in an edit so it may be missed), is that the German vehicles have better optics.

That's why captured vehicles should be used. smile.gif

Maybe I'm missing the point, but I thought only German vehicles had different types of optics modeled, and the other nationalities (even using captured equipment) only had the same "standard" level of optics. So, a German Panther will have better optics than a Russian captured Panther. True?

[Edited to practice my spelling skills.]

[ October 30, 2002, 04:15 PM: Message edited by: Ace Pilot ]

Link to comment
Share on other sites

Originally posted by Cameroon:

One flaw that I see in the majority of these test scenarios (I commented on it above, but in an edit so it may be missed), is that the German vehicles have better optics.

That's why captured vehicles should be used. smile.gif

It actually will not matter because we are looking how the AI and a human perform in the exact same situation. We could look at T-26s vs. King Tigers and still get comparable results.

(ok, ok, we couldn't. Really both tanks have to be able to kill each other to get meaningful results.)

--Chris

Link to comment
Share on other sites

Originally posted by Maastrictian:

I also, am quite studiously not claiming anything smile.gif In fact, based on the 176 test I've run I'd say any difference is pretty small, if present at all.

Warren -- try right clicking on the link I gave and choosing "save as". That works for me at least.

Cameroon -- You are welcome to do whatever tests you want and post the results, but I think we wil all get farther if we all concentrate on using the same testing scenario, varrying only by the targeting orders we give. That way we can combine all our results and see what we get, rather than having 20 different tests testing different things none of which is significant by itself.

I strongly agree with Madmatt's comment that we need more than 100 tests to get anywhere, preferably, in my mind, more than 1000. I can perform that many, but it will take me a week. It will be much quicker if everyone simply runs my scenario once or twice.

(note, I'm not at all hung up about whose scenario gets run. Its just that I'm the only one whose posted a scenario for others to run at this point.)

--Chris

I agree that it would be much better if the same scenario were run. However, as I said, I can't agree that using Stugs vs T-34s is a good study. The Stugs have both an optics and, I think, a silhouette advantage. I think both of those are serious issues. I can put my scenario up in a few minutes if people want to run it.
Link to comment
Share on other sites

Originally posted by Ace Pilot:

</font><blockquote>quote:</font><hr />Originally posted by Cameroon:

One flaw that I see in the majority of these test scenarios (I commented on it above, but in an edit so it may be missed), is that the German vehicles have better optics.

That's why captured vehicles should be used. smile.gif

Maybe I'm missing the point, but I thought only German vehicles had different types of optics modeled, and the other nationalities (even using captured equipment) only had the same "standard" level of optics. So, a German Panther will have better optics than a Russian captured Panther. True?

[Edited to practice my spelling skills.]</font>

Link to comment
Share on other sites

Originally posted by Maastrictian:

</font><blockquote>quote:</font><hr />Originally posted by Cameroon:

One flaw that I see in the majority of these test scenarios (I commented on it above, but in an edit so it may be missed), is that the German vehicles have better optics.

That's why captured vehicles should be used. smile.gif

It actually will not matter because we are looking how the AI and a human perform in the exact same situation. We could look at T-26s vs. King Tigers and still get comparable results.

(ok, ok, we couldn't. Really both tanks have to be able to kill each other to get meaningful results.)

--Chris</font>

Link to comment
Share on other sites

For anyone who wishes to run my test scenario, here it is.

It is designed to be used in the testing of 1st shot statistics. I am ignoring the aspect of multiple shots because, with vehicles that are so likely to kill one another, the first shot is very important. Furthermore, the contention of this thread is such that the "1st shot" of the AI was "enhanced."

Personally I feel that the various tests haven't been rigorous enough ;) So here goes for more stats recording. Please, I'm not knocking anyone on this either. I just think not enough variables were controlled with the others.

Link to comment
Share on other sites

Originally posted by Cameroon:

Ok, I see what you're saying, but I still believe that using the same vehicle will result in more 'sound' results. Controlling as many variables as possible smile.gif

That works for me. It will probably quiet some of the detractors who want one more thing to pick at too smile.gif . If you have time to make the scenario I'll run it until the cows come home, and I hope others will too.

I'd encourage you to:

1) Use as many tanks as possible. On the order of 50.

2) Make sure none of them have smoke or T or any other "special" ammo.

3) Put all of them in rough ground so they don't move.

4) Remove any flags so those aren't influencing the battle somehow

5) Use T-34s as they don't have smoke dischargers.

6) Add a bunch of units away from the battle so global morale plays a minimal role. Pillboxes are good as they have very few polygons.

If you need hosting space, or a mirror I can put it up on my site. E-mail it to me at dinosaur@noct.net.

--Chris

Link to comment
Share on other sites

Originally posted by Maastrictian:

</font><blockquote>quote:</font><hr />Originally posted by Cameroon:

Ok, I see what you're saying, but I still believe that using the same vehicle will result in more 'sound' results. Controlling as many variables as possible smile.gif

That works for me. It will probably quiet some of the detractors who want one more thing to pick at too smile.gif . If you have time to make the scenario I'll run it until the cows come home, and I hope others will too.

I'd encourage you to:

1) Use as many tanks as possible. On the order of 50.

2) Make sure none of them have smoke or T or any other "special" ammo.

3) Put all of them in rough ground so they don't move.

4) Remove any flags so those aren't influencing the battle somehow

5) Use T-34s as they don't have smoke dischargers.

6) Add a bunch of units away from the battle so global morale plays a minimal role. Pillboxes are good as they have very few polygons.

If you need hosting space, or a mirror I can put it up on my site. E-mail it to me at dinosaur@noct.net.

--Chris</font>

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.

×
×
  • Create New...