legend42 Posted July 19, 2007 Share Posted July 19, 2007 I've always wondered how that happened with all the thousands of hours of play testing?Is it that more issues pop up in the hands of thousands vs twenty testers?Yet there is already a patch before Shock Force reaches the masses. 0 Quote Link to comment Share on other sites More sharing options...
gibsonm Posted July 19, 2007 Share Posted July 19, 2007 Well with a limited, all volunteer group doing it in its spare time, like us its bound to happen. Just think how many bugs MicroSloth lets through and it has a paid, dedicated bunch of testers. 0 Quote Link to comment Share on other sites More sharing options...
jeffsmith Posted July 19, 2007 Share Posted July 19, 2007 Doesn't necessarily have to be a Bug does it ? Could be a feature that needs tweaking or just an improvement in gameplay 0 Quote Link to comment Share on other sites More sharing options...
Capt. Toleran Posted July 19, 2007 Share Posted July 19, 2007 Haha, well, I'm sure plenty of us would have loved to volunteer to help, all you need to do is ask... Seriously, though I understand that many QA functions require long and repetitive testing with log-taking, certain issues (like graphics) could be tested with a non-functional engine as a download for comparison on multiple systems. It also would have given us eye candy to drool over. Maybe something to think about for future games. 0 Quote Link to comment Share on other sites More sharing options...
ChrisND Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by legend42: I've always wondered how that happened with all the thousands of hours of play testing?Is it that more issues pop up in the hands of thousands vs twenty testers?Yet there is already a patch before Shock Force reaches the masses. We purposely ignore bugs during those thousands of hours of playtesting just to piss players off. 0 Quote Link to comment Share on other sites More sharing options...
dan/california Posted July 19, 2007 Share Posted July 19, 2007 I read a really good piece about this in another game forum, and it said that what most in house testing looked for was actual show stoppers, things that crashed it to desktop or worse. And this was a company that dwarfs Battlefront. These are also the things that make customers completely loony,even the casual ones. An unnecessary step with most of the folks on this forum. The subtle stuff does not come out until thousands and thousands of people play the game and notice that RPG-7s are killing far more Abrams than they ought to or some types of infantry with no thermals can see through smoke. 1.01 will be playable. About 1.05 it will be spectacular, its just part of it. 0 Quote Link to comment Share on other sites More sharing options...
MikeyD Posted July 19, 2007 Share Posted July 19, 2007 Three things usually. 1) The bug's in some obscure corner that nobody visits (how often do YOU play in Hot Seat Basic training mode?) 2) Second is more embarrassing - everyone knows about some little bug and everyone assumes someone else reported it! 3) This one's the cruelist. Game's all put to bed clean-as-a-whistle. But just before release some innocent last minute alteration is made during final housecleaning and BAM! Like being whacked in the face with a big cream pie! :eek: Oh, there's a number 4 too. After living with XX builds of the game for X straight months some stuff that you'd class as a bug we don't even notice anymore! Like when you visit greatgrandma's house and you open the door to get that first whiff of 'old lady apartment smell'. But these are discussions about 'how the sausage gets made'. Really, its best you don't know how the sausage gets made. [ July 18, 2007, 09:26 PM: Message edited by: MikeyD ] 0 Quote Link to comment Share on other sites More sharing options...
Michael Dorosh Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by MikeyD: Three things usually. 1) The bug is in some obscure corner that nobody ever visits (how often do YOU play in Hot Seat Basic training mode?) . It doesn't have to be obscure, either. The beta team is weighted heavily with real world military experience, so testing of functions tends to be in that area. Gamers pick up the product and do things - "gamey" things - that beta testers from that background might not even think about doing. "Hey, how come my tanks turn purple when I try to fire a missile through an open window at point blank range?" 0 Quote Link to comment Share on other sites More sharing options...
Achim Posted July 19, 2007 Share Posted July 19, 2007 Its hard to spot bugs. As a software engineer you can use "use cases" to get thinks working. What will the user do and what should happen. But the user is a complex thing, he dont use your use cases. He always try other ways than you excpected and your realy think about what he will do. I think there are two kind of bugs. The realy angry bugs (software doesnt start, game crashes after i press the "let´s go button"). This one should not appear and the programmer should find them with tests. And the "My tank turns green if i fire through a window at point blank and loaded infantrie on my tank that throws with eggs on the other tank" bugs. This bugs can be found by beta testers. But its realy unlikely. I think no programmer realy say "Today, i will put this two nice bugs in my software". At the end the user is always a tester. The only thing i expect is, that the bugs get fixed soon after they reported. This sounds basic, but as a comany u cant let your programmer wait for bugs and do nothing, they should work on other projects and make money for the company (bug fixing burns money). And disrubt the new project to fix bugs for an old project burns money too. The only way to solve the problem is to write perfect software without any bugs. Expect the game in the year 2050 and the company needs 500 000 professional tester. The game only costs 9999.99 $ (shipping is free, because we beam up things in the year 2050). And u need a "new" computer, because on the new stuff the game doesnt run, there isnt a trench for this silver disc to put in. The new computers works direct connected with your brain and there are only a few holes in your body u can stick this silver thing (but the marketing branch dont want that sollution ). 0 Quote Link to comment Share on other sites More sharing options...
PFMM Posted July 19, 2007 Share Posted July 19, 2007 Complicated designs create complicated prablems with complicated solutions. And the solutions don't work all the time in every instance. And thousands of hours is not enough to uncover every instance in a complicated design. 0 Quote Link to comment Share on other sites More sharing options...
RabidRabbit Posted July 19, 2007 Share Posted July 19, 2007 What about technical bugs such as the game having issues with the 8800 video card drivers as claimed by one of the reviewers. None of the beta testers had an 8800 card? 0 Quote Link to comment Share on other sites More sharing options...
JaguarUSF Posted July 19, 2007 Share Posted July 19, 2007 Not enough combinations of hardware for testers, as noted by the 8800 deal. 0 Quote Link to comment Share on other sites More sharing options...
Bunyip Posted July 19, 2007 Share Posted July 19, 2007 After looking at something for the nth time your brain is no longer learning about it, it is now making subconscious assumptions about something it recognizes, hence the tester fails to see the bug that crept in with the last build. In this case the more hours of testing the more likely the bug is to slip through. 0 Quote Link to comment Share on other sites More sharing options...
SGT_56M Posted July 19, 2007 Share Posted July 19, 2007 I have an 8800 GTS and have no issues with the card. Granted, I only have one setup with the card on one OS with one combination of hardware. Seriously though, I have over 10 years of software QA experience, much of that within the game industry. Establishing test cases for an open ended and diverse program such as a game is infinitely difficult more difficult that for a piece of software that will only run on a single set of hardware and perform a single set task. And the dynamic goes well beyond the confines of the software use as it ties in the HUGE amount of hardware and software configurations and settings any one user has with their computer. PC’s are notoriously difficult to test games on, let alone any program. Another consideration you must realize is that game developers do not have an infinite amount of time to bring their program to market. Budgets and cash reserves must be able to last as long as it takes to get the game working. Blizzard is good about this, but they release very few titles and have copious amounts of cash on hand to let a game stew for ages. Yet there are still issues with their products. Someone posted elsewhere also that the pressing of the actual CD begins to take place weeks before a release. So zero day patches are now very common as the internet is very helpful in getting that out to users the day they receive their copy. So it all boils down to finding the proper medium between a “complete” game, schedules, budgets and the impact any one issue has with the overall playability of the game. It’s an art more than a science. A senior game developer once explained it as, “Imagine the game as a lump of jello. You get it to a point where it is not wiggling too much and ship it. Then if it starts to wiggle after release, you do what you can to get it stabilized more.” If you waited for there to be no bugs at all in any shape or form, you would most likely be waiting for a piece of vaporware. 0 Quote Link to comment Share on other sites More sharing options...
Dirtweasle Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by Michael Dorosh: It doesn't have to be obscure, either. The beta team is weighted heavily with real world military experience,... Probably then that makes the case they should add a few "Joe Average" civvie types as well as try and recruit a "new to the genre" type. 0 Quote Link to comment Share on other sites More sharing options...
Redwolf Posted July 19, 2007 Share Posted July 19, 2007 Beta testers are not totally representative of the game audience. The real game audience extends further on both ends, people with extreme attention to details and people who fatfinger everything and can make everything crash. Although beta tester programs try to recruit people that aren't only "hardcore" gamers, these two groups are usually too hard to work with for the beta test leads. Even if you are willing to work with them they might not be interested in investing hundreds of hours for a free copy of the game. In addition, people in the real world have really screwed up PCs and network connections and those who volunteer for beta testing don't. 0 Quote Link to comment Share on other sites More sharing options...
Sergei Posted July 19, 2007 Share Posted July 19, 2007 I'm reminded of the "fortifications count as losses in AAR" bug in CMBB. It was years after the release that someone finally noticed it. Testers of course have only a limited amount of time per week that they can spend playing and then writing on things they've noticed. Then the programmer only has a limited amount of time per week trying to understand what the heck those bug reports are about, and how to repeat it, and what causes it, and to fix the cause. And the response from testers could be "it's still broken", so it can take a few tries before all are satisfied. Meanwhile the clock is ticking and people in the forums growl. Additionally the programmer is continuously implementing new features which need to be tried out. Once the deadline is near there's going to be a big hurry to spot, test and fix all the latest bugs. But since there's anyway a delay between the game going gold and it hitting the market, it's reasonable to use that gap for getting a patch ready for release day. 0 Quote Link to comment Share on other sites More sharing options...
flamingknives Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by Dirtweasle: </font><blockquote>quote:</font><hr />Originally posted by Michael Dorosh: It doesn't have to be obscure, either. The beta team is weighted heavily with real world military experience,... Probably then that makes the case they should add a few "Joe Average" civvie types as well as try and recruit a "new to the genre" type. </font> 0 Quote Link to comment Share on other sites More sharing options...
BFCElvis Posted July 19, 2007 Share Posted July 19, 2007 Dirtweasle, can't speak of the other testers but I am your Joe Average. No knowledge of modern warfare, no military experience and I don't spend much time designing scenarios (you will my my name on the credits of none). But I play the hell out of the game. And as I told a fellow tester recently "our job is to break it" and I've been doing as good of job of that as I can. 0 Quote Link to comment Share on other sites More sharing options...
SGT_56M Posted July 19, 2007 Share Posted July 19, 2007 I think you told me that on the phone last week. 0 Quote Link to comment Share on other sites More sharing options...
aka_tom_w Posted July 19, 2007 Share Posted July 19, 2007 This is why I like beta testing: " The beta team is weighted heavily with real world military experience, so testing of functions tends to be in that area. Gamers pick up the product and do things - "gamey" things - that beta testers from that background might not even think about doing. " Gamey things, trust me I am there for one reason only, to break things that shouldn't break, and complain about how my gamey exploits work or should not work. Don't worry there are gamey/computer geek video game testers there too. Some stuff is reported as a bug and well, maybe it is not a bug but an "undocumented feature" (Limitless spotting rounds in CMBO comes to mind, some gamey gamer figured out you could never run out of spoting rounds, as they did not originally count against your limited number of rounds or arty available, so yes bad example of an undocumented feature, THAT one was a bug) As with all BFC CMx1 and now CMx2 products you can count on post release patches to fix new bugs when they become glaring and obvious and need attention. I would not worry too much about bugs and beta testing, my sense is the whole thing is completely under control. As for breaking things Elvis and I like to break things together whenever we can like um oh lets see.... what takes 2 crazy beta testers to break it? Um PBEM and TCP/IP (Yup we have tested both) (...oh and I am A MAC guy running on a borrowed PC windoze laptop with possibly the least amount of RAM of any beta Tester, 512 megs) [ July 19, 2007, 01:49 PM: Message edited by: aka_tom_w ] 0 Quote Link to comment Share on other sites More sharing options...
MikeyD Posted July 19, 2007 Share Posted July 19, 2007 Ouch Tom, you're only 512 megs? You are definitely their benchmark 'minimim capabilities tester', a post i held while Beta testing CMAK. I just saw a TV newsmagazine report about some monster computer gaming design house inhabiting a shiney skyscraper on the Vancouver BC waterfront. I wonder how big and shiney BFC's downtown skyscraper office is! 0 Quote Link to comment Share on other sites More sharing options...
flamingknives Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by aka_tom_w: (...oh and I am A MAC guy running on a borrowed PC windoze laptop with possibly the least amount of RAM of any beta Tester, 512 megs) Yeah, rub it in. 0 Quote Link to comment Share on other sites More sharing options...
aka_tom_w Posted July 19, 2007 Share Posted July 19, 2007 would that be Electronic Arts??? They are HUGE! 0 Quote Link to comment Share on other sites More sharing options...
aka_tom_w Posted July 19, 2007 Share Posted July 19, 2007 Originally posted by MikeyD: Ouch Tom, you're only 512 megs? You are definitely their benchmark 'minimim capabilities tester', a post i held while Beta testing CMAK.Yup that's me, I am bordering a "special needs" computer user, and I don't really know windows XP all that well either, however I have learned a TON about what makes windows work testing and playing the game in windows everyday now. 0 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.