How do Bugs get past Beta Testers?

[Sitting Duck]

Here's how you create the most perfect software possible.

http://www.sinz.org/Michael.Sinz/Software.html

[aka_tom_w]

Thanks Sitting Duck!

That was a GOOD read!

Entertaining and enlightening

[Battlefront.com]

Seems things have been covered well enough, but I'll add in a few cents worth.

As was said earlier, even with paid testing services (that cost more than you can imgine!), things get through. The number of bugs that get through is somewhat proportional to the size, complexity, and type of software. The larger, more complex, and harder it hits the hardware the greater the total number of bugs, therefore the great the number that slip through.

When I worked at Impressions (Sierra) we released a HUGE game that I worked on. We had something like 1000 documented bugs that were not fixed when the game shipped. These were all the way from "there is an extra white pixel in the 8th screen in on that feature nobody uses" to "if the dude goes to the left when two dudes right next to him go right, then 6 battles later the game will crash. Or so we think, because that's the time we saw it." After release we fixed a bunch and the rest remained unfixed. Nobody who noticed complained, nobody who complained stopped playing the game. Reviews were awesome, sales stellar, an expansion pack was made that sold well, and everybody was happy (well, as happy as gamers can ever be!)

Think about it... 1000 (mostly) obscure bugs at even 10 minutes each comes out to be almost a month of normal 8 hour, 5 day a week work days. And when fixing those 1000 bugs the programmer/s will likely introduce another 200 At some point you just have to say "it's done" and move on. The market place will judge if it was too early to move on or not.

Steve

[Berlichtingen]

From my experience, the method that BFC uses lets through far fewer bugs than you will see in much larger software companies. Without going into detail, the method is very methodical and well thought out. However, no method of beta testing software will catch everything. As one of my programing instructors told me centuries ago, if architecture were like computer programing, civilization would collapse when the first woodpecker came along.

[Dirtweasle]

Originally posted by Berlichtingen:

From my experience, the method that BFC uses lets through far fewer bugs than you will see in much larger software companies...

Have to agree from what I've seen with the past CMX1 releases, especially when compared to other products in the same genre.

The point I was trying to make, and was answered by Elvis (God now that is funny to me!), is in my experience you always need that random loosey-goosey type that is going to do "weird" stuff with the game.

Anyways, hope a few of your scenario designs either made it into the game Berli or are going to be hosted someplace or another ratehr soon.

[Berlichtingen]

Originally posted by Dirtweasle:

Anyways, hope a few of your scenario designs either made it into the game Berli or are going to be hosted someplace or another ratehr soon.

I didn't particiapte in this beta test. I did preorder the game, but I'll be honest, the subject matter hopes zero interest for me. I have no interest in the modern US Army, nor the Middle East. I'll probably build a scenario or two once I get it

[Dirtweasle]

Well, maybe I'll try and port over that scenario of yours where the Russians end up surrounding the German tanks in the snow if you don't mind. Only way I could ever beat that one was to sucker the AI into a premature cease-fire.

[Berlichtingen]

Feel free Dirtweasle. I figure anything I do will be new stuff rather than remakes. Maybe some stuff based on Chechnya.