AI cheats! (with real data)

[CavalryMan]

Bump. Only because there is no way that so much work should be allowed to fall off the bottom before the next natural number has been attained. Also I would hate to think of all the thousands of VR tankers lives having been spent in vain.

Keep up the good work, dont understand it but it passes the weight test

PS isnt chi square a Martial(or Marital) art

[Treeburst155]

Warren, I have your scenario. I also have some questions. How are you counting vehicles that rout off the map? What about crews actually in the act of "bailing out"?

If I'm going to be counting survivors I will need to define exactly what it is that constitutes a survivor. It probably doesn't matter if I define the term a little different than you as long as I am consistent throughout all my tests. Can I trap the vehicles on the map, or is freedom of movement something you want to allow?

Treeburst155 out.

[Captain Wacky]

All this spinning hurts my neck. Boy am I glad I copped out of my math credit requirement and signed up for the formal logic philosophy course instead of stat

[Warren Peace]

Treeburst,

I counted vehicles that routed off the map as kills. I never encountered crews in the act of bailing by the time the third turn was over. They either have bailed by then or their not going to.

Warren

[Treeburst155]

I played Warren's 6-lane scenario 20 times from each side. I targetted manually at every opportunity.

Human as Germans:

82 surviving German vehicles

32 surviving Russian vehicles

Human as Russians:

74 surviving German vehicles

37 surviving Russian vehicles

The best performance for each side occurred while under human control.

Treeburst155 out.

[zukkov]

anyway, i think the most important thing i learned here is that manual targeting is faster than ai targeting. right guys? i learned that didn't i? guys? lol...

[Lt. Kije]

The best performance for each side occurred while under human control.

Treeburst155 out.

Wooooooooo......

Was not expecting this!

-- Lt. Kije

Scorekeeper and Historian

[ November 01, 2002, 10:58 PM: Message edited by: Lt. Kije ]

[Treeburst155]

Zukkov,

Yeah, you definitely want to manually target in AFV duels.

Lt. Kije,

Yeah, how about that. I'm probably doing something different than he is. Perhaps I have an outlier result? It would take quite awhile to get a huge number of samples with this test. You have 120 duels from each side right now.

Treeburst155 out.

[Warren Peace]

Treeburst:

I played tonight as well. Six times on each side.

Human as Axis

11 Stugs vs. 26 T34

Human vs. Allies

12 Stugs vs. 25 T34

Guess what? I didn't see any difference! I also set up a battle where instead of STugs I used captured t34/85.

As Germans

21:21

As Russians

20:20

I guess my earlier observations were just a fluke

Oh well, thats how science works sometimes

PS Kind of anticlimatic, but boy what a great thread!

[Broken]

Jeez, you guys still at this?

As I remember, in CMBO, there were some threads discussing the idea that there were hidden variables for every unit created. For example, some units are created as "fanatics", but you don't know which ones. If there are hidden variables effecting gunnery accuracy, it might explain some of the descrepancies you are seeing.

A unit's hidden variables are set when the unit is purchased, IIRC. So it doesn't matter how many times you run a test; if you haven't purchased new units, you are stuck with the same hidden variables. To do a test properly under these circumstances, you must delete and repurchase new units every time, to get a properly random mix.

Just thought I would put that fly in the ointment.

[Treeburst155]

Very interesting, Broken! I'll have to experiment with that some day.

Warren,

I ran 10 more from each side just for kicks.

Human as Germans:

35 German survivors

19 Russian survivors

AI as Germans:

38 German survivors

19 Russian survivors

Virtually the same.

Yeah, this has been a great thread. There's still an interesting thing I might look into someday:

The first shot among equal crews in the same vehicles will go to the human player the vast majority of the time if he manually targets, no matter which side he plays.

I tested this a bit last night. It's true.

Treeburst155 out.

[ November 01, 2002, 11:57 PM: Message edited by: Treeburst155 ]

[Lt. Kije]

Oh my ears and whiskers!

What a remarkable outcome!

-- Lt. Kije

[Treeburst155]

Now, where was I two days ago, before this thread took over my life? Oh yeah, that's right. I was going to take a nap.

Treeburst155 out.

[Seanachai]

Originally posted by Treeburst155:

Now, where was I two days ago, before this thread took over my life? Oh yeah, that's right. I was going to take a nap.

Treeburst155 out.

Well done, lads.

Now, do us all a favour and sort out the way weather works, eh?

Shouldn't prove much challenge for you lot...

[Ruthless]

Ok, I really didn't want to get involved in this extreme statistics thread (other than lurking on it all last week at work), but my wife has a question she needs answered by a statistician for one of her graduate biology courses:

How do you determine what the "hypothesized mean difference" should be for a paired two sample for mean abundance t-test? This is just slightly off the wall, but you guys seem to be statistics experts, and my husband has been reading me selected passages. Thanks!

[Treeburst155]

Um.....that's outside the realm of my "Blue Collar Statistics". Perhaps one of the pros will be along shortly to help you out. If you're real unlucky a whole bunch of them will try to be helpful. In that case, you will end up waiting for several days while they fight it out.

Treeburst155 out.

[sinclair]

Originally posted by CMplayer:

Your sample is way to small to draw those kinds of conclusions.

I saw the same thing...

The AI has better chance to hit

I did a mirror 10 times on a game...

AI hit after 2 or 3 fire...

ME??? betwen 5 and 10... If my tank is not destroy...

[Ruthless]

Wife again:

She thinks she should be using the Mann-Whitney U Test instead of a T Test...

she needs to do this in Excel.

Ruthless again:

Ok, I know all you die-hard statistics fanatics out there can crunch WWII video game numbers, so you should be able to help her out...? Get to it, lads!

Treeburst, thanks for running thousands upon thousands of tests. Being a software engineer myself (specifically I do real-time embedded systems for avionics on fighters) and working, in my spare time, on a warsim myself (you can preeorder if you want to-- if I ever finish, it may be done in a decade or so during which time I will happily keep your money safe in my bank account) I am interested in this thread and other threads about all the inconsistencies between sim and real life as well as discussions about the AI's, etc. etc.

Keep 'em comin'...

[GWDWD]

Love it!

I'm in the middle of an intermediate-level statistics course and almost posted (in the general forum, naturally) a question about Bayes' Theorem a month or so ago. I can do the calculations but I was/am having trouble understanding what it really means.

"But who would know THAT sort of thing?", I asked.

Guy

P.S. If anyone can explain Bayes' Theorem to me, please do!!

[ November 03, 2002, 12:38 AM: Message edited by: GWDWD ]

[Lt. Kije]

Hee, hee. My first major academic publication was on ways to explain Bayesian reasoning using graphical representations.

Jumpin Blue Tamales, I never thought this stuff would come up in the context of a wargame.

Sadly, it would be pretty hard to post a diagram here.

Basically, as you know, Bayesian reasoning (Thomas Bayes was a minor clergyman in Tunbridge Wells, England. I have on my bookshelf a photo of his burial crypt) is about moving from a starting probability that something is true to a posterior probability it is true, based on new information.

It is notoriously difficult to do intuitively and a classic study by Casscells, Schoenberger and Grayboys showed that Harvard physicans screw up badly when you give them a simple reasoning problem involving a positive medical test screening for breast cancer. Danny Kahneman, who just won a Nobel prize, had as one of his earliest famous papers a more general demonstration of the failure of ordinary people to be able to do this seemingly simple reasoning.

The key, argued Kahneman and Tversky (Amos Tversky, the real brains behind the Kahneman and Tversky papers, died a few years ago and so he couldn't get the Nobel), was inability to take into account 'base rate', called 'prevalence' in the medical world. In fact, Kaheman and Tversky were wrong, as your own Lt. Kije later demonstrated. The true inability is knowing how to make use of false positives.

Here, try the classic Casscells et al problem.

35 year old woman comes in for a physical, a routine screening test comes back positive. Breast cancer test. As with all tests, there are three key parameters.

1. We know that women who really do have breast cancer test positive 95% of the time.

2. We know that women who really do not have breast cancer test negative 90% of the time.

3. In 35 year old women, about one in a thousand has breast cancer. (This is the 'base rate', or 'prevalence'.)

What is the probability, after the test has come back positive this woman has breast cancer?

Would you like some thinking music?

-- Lt. Kije

(I bet this thread gets locked or moved pretty doggone soon. It has drifted so far off topic that, well, it is pretty off topic.)

[ November 03, 2002, 12:53 AM: Message edited by: Lt. Kije ]

[GWDWD]

Lt. Kije,

Is it fair to say that Bayes' Theorem is a way to go from sensitivity, specificity, and prevalence to positive predictive value and negative predictive value? That's kinda what I've been thinking...

Dammit, I love statistics.

Guy

[Lt. Kije]

GWDWD said:

Is it fair to say that Bayes' Theorem is a way to go from sensitivity, specificity, and prevalence to positive predictive value and negative predictive value?

Yes, exactly. Although there's a long chasm between being able to say that and being able to intuitively understand Bayesian reasoning. How do we reason our way from A to C here?

A. Prior probability

B. New information

C. Posterior probability (in light of new info)

As you appear to know and understand, predictive value means posterior probability. If the test came back positive, the likelihood the woman has the disease is called 'positive predictive value', a posterior probability. If it came back negative, the prob the woman does not have the disease is called 'negative predictive value'.

-- Lt. Kije

A pig in sh*t

[Sten]

Heh. Of all places to stumble upon that old fave problem.

The answer is just below 2% chance (1.85) if I remember correctly.

This is hard enough to realize for professionals, but explaining it to patients is harder still.

The best way is usually to explain that there is a 98 percent chance that they do not have the cancer or whatever.

When given time to reflect on the matter and being calmed down by the ruling out the diagnosis by other means, very few come back with the clever question "What kind of crappy test was that?" But those who do usually appreciate the full answer.

Sten

[Tarquelne]

inability to take into account 'base rate', called 'prevalence' in the medical world. In fact, Kaheman and Tversky were wrong, as your own Lt. Kije later demonstrated. The true inability is knowing how to make use of false positives.

How interesting all this discussion of the nitty gritty of CM's engine is! I think it'd help to know what Lt. Kije's basic argument in regard to false positive vrs. base-rate error is, or possibly even the relevent publication information.

I ask this not out of any off-topic interest, but _only_ to help ensure than a furor caused by false positives in any future CM gunnery tests is avoided.

I wouldn't dream of posting anything off topic.

[russellmz]

Originally posted by Sten:

Heh. Of all places to stumble upon that old fave problem.

The answer is just below 2% chance (1.85) if I remember correctly.

This is hard enough to realize for professionals, but explaining it to patients is harder still.

The best way is usually to explain that there is a 98 percent chance that they do not have the cancer or whatever.

When given time to reflect on the matter and being calmed down by the ruling out the diagnosis by other means, very few come back with the clever question "What kind of crappy test was that?" But those who do usually appreciate the full answer.

Sten

like the man above said: