Is CM dying?

[BletchleyGeek]

I don't think the open beta model would work well with BFC's current DRM system - things could get messy with activations and deactivations (say, you install a beta patch and you want to revert to a 'official' patch, just the first thing that comes to my mind). To be honest, not even Matrix release procedures are agile enough: Steam does and it is indeed leading the way in delivery, completion and sensible DRM policies.

I'd suggest BFC to consider getting their products on Steam as a secondary shopfront. Yes, you'll have to pay a fee for that to Valve, but on the other hand, you can give away as many Steam keys as you want (somebody who buys from you can redeem an Steam key) for free.

@warrenpeace et al. Anybody promising you that a software product is bug-free is selling you smoke (or smoky snake oil, if you want). There are always bugs, some of them are noticed, some of them aren't. And the regular Joe, in his forties, from Winnetka, Illinois, with a keen interest in history and wargaming, probably hasn't been trained in Q&A: which means learning how to collect data, analyze it and make sense out of it.

Most (99%) of the beta-testers you'll find around the wargaming world are well-meaning individuals that probably devote a substantial part of their spare time to check that installers work, games don't crash, models look good, spotting gross ubermodelling problems, etc. There are other things that require much more time in order to be properly identified and possibly require extensive analysis of test runs over benchmarks, and contrasting those results on readily available sources on arcane topics such as as battle psychology or ballistics.

The great thing of open betas is that individuals with great deals of time in their hands devote that time to do the kind of painstakingly patient and devoted work that I describe above. Which a great majority of the population find to be a royal pain in the a**. This pool of 'well-educated individuals with lots of time in their hands willing to do hard work for free' is going to become a major asset, if it's not already.

[AlexUK]

Speaking as someone who has only got involved in a very limited amount of software testing on very simple online payments project - you design a script to test, you identify some bugs, you then need to re-run everything, because sure enough, fixing something sometimes breaks something else that was working together, you then fix that thing and re-run everything all over again. Each round of testing takes a long time.

Now scaling that up to a game like CM would drive me round the bend. During the c3k/Bil battle (and boy, do I want to see how that one plus out), several bugs were identified that had subsequently been fixed for the release, so I am guessing that it is not like they did inadequate testing on the game and missed some things - I think it is more like they did loads of testing, found and fixed loads of bugs, and made a call on release on the basis of further testing. There must come a point where you have to make a decision to release.

Also, the more time they spend on the patch, the less time they spend on EF and modern....

And they are hard at work on the small bugs that got through - one very happy customer here - now if I could only stop getting hammered in Die Letze campaign, life would be all roses

Anyone read Sniper One yet? I'm getting all evangelical about it.

[sburke]

As a beta tester I just want to say thanks for the implied suggestion that we are all incompetent and missed blatantly obvious stuff that some other group would have found no problem with actually no idea of the volume of stuff we have already caught or were testing. yep thanks. Command Ops and War in the East as I recall were released with no obvious bugs whatsoever. Thank god I wasn't working on it, who knows what it might have looked like.

Oh wait a minute, I just looked at their 2012 update, 2 years after release that has... wait for it.... bug fixes!!!! Okay which one of us was working on that project? MikeyD? AKD? C'mon guys own up!!!

Note their first bug fix patch for WitE was released within 4 days. Don't get me wrong, I am not criticizing those guys, I bought both those games and like them. However to imply somehow their process eliminated blatant bugs getting through is a disservice to them and BF. Software is what it is.

[BletchleyGeek]

As a beta tester I just want to say thanks for the implied suggestion that we are all incompetent and missed blatantly obvious stuff that some other group would have found no problem with actually no idea of the volume of stuff we have already caught or were testing. yep thanks. Command Ops and War in the East as I recall were released with no obvious bugs whatsoever. Thank god I wasn't working on it, who knows what it might have looked like.

And here comes sburke, which doesn't read the posts and thinks everyone but himself is an a**hole. Let me quote myself and highlight something I wrote and I think it's quite relevant to:

Most (99%) of the beta-testers you'll find around the wargaming world are well-meaning individuals that probably devote a substantial part of their spare time to check that installers work, games don't crash, models look good, spotting gross ubermodelling problems, etc. There are other things that require much more time in order to be properly identified and possibly require extensive analysis of test runs over benchmarks, and contrasting those results on readily available sources on arcane topics such as as battle psychology or ballistics

See what AlexUK says, and I don't think I need to further embarrass you pointing out to the thread where one guy probably detected a problem with bad plate data on the JagdPanzer lower hull. Just to mention one example.

[akd]

I don't think I need to further embarrass you pointing out to the thread where one guy probably detected a problem with bad plate data on the JagdPanzer lower hull. Just to mention one example.

That was not bad plate data. It was a subjective call on assigning an armor quality rating. The change made is minor bump up in quality.

[LukeFF]

ANYTHING we release comes with an implicit acknowledgement that there may still be some errors in it. Even the fix to the fix to a fix for a fix. It's just the way software works. Anybody who thinks it can ever be anything other than that is delusional.

Exactly. I've been an internal beta tester for Rise of Flight for about 2 years now, and it never fails that some sort of new bug crops up in the release build and isn't detected until the patch has gone live. And then, of course, all us testers look at each other and ask, "How did we miss that?" It's human nature. It happens.

[warrenpeace]

Just to clarify, I certainly wasn't implying any criticism of the beta team. I'm sure they caught plenty of bugs. I was just stating what I perceive as the benefits of an open beta, which are used by a lot wargame makers. I certainly respect BFC's decision not to go that way, but personally I kind of like it.

As long as the annoying little bugs get fixed in a timely manner (which it sounds like the will be) I am happy.

[BletchleyGeek]

That was not bad plate data. It was a subjective call on assigning an armor quality rating. The change made is minor bump up in quality.

That's interesting: so CMx2 is actually modelling not only thickness but also manufacturing quality? That kind of confirms that CMx2 succeeds at being featuring very high-fidelity models and hiding those details from the average player.

[sburke]

And here comes sburke, which doesn't read the posts and thinks everyone but himself is an a**hole. Let me quote myself and highlight something I wrote and I think it's quite relevant to:

See what AlexUK says, and I don't think I need to further embarrass you pointing out to the thread where one guy probably detected a problem with bad plate data on the JagdPanzer lower hull. Just to mention one example.

Actually Bletchley I was not responding to your post. I also was not saying we don't miss anything, hell we missed the MG model issue. Honestly it is a bit embarrassing and I have actually previously apologized personally on this forum not too long ago on another issue. Yes we miss stuff, sometimes really blatant stuff.

My response was to the argument for a change based on another model that supposedly would not have that issue. The specific example of that model was matrix's beta programs for WitE and Command Ops. WitE had a patch within 4 days- 4 days is an incredibly short window to develop, test and release a patch. I would frankly be surprised to hear that patch wasn't actually in development prior to the release of a product that had known issues. A judgement call had to be made at some point though as to when is good enough. Again that isn't a critique of those guys but a recognition that software is not like building furniture. It's never really complete and in BF's case may never be till they decide they just aren't doing it anymore. Looking at the high dollar big game company releases I have seen bugs that make BF look absolutely stellar about their QA. Within the constraints of their resources and economic model I'd have to say BF does a pretty darn good job. Perfect- no, but better dollar for dollar than most of the stuff I have seen out there.

And I don't think everyone but myself is an ***hole. We are all **holes at one point or another. It's part of being human.

[Collingwood]

Speaking of command ops, the dev recently stated he can't keep squashing bugs/patching the game without remuneration. So he'll soon stop releasing patches for the current game, and start work on the new version. It's a good game with excellent AI - but sounds like it will never be "bug free", nor finished so to speak.

As a programmer I have an insight into the issue of bugs vs release vs testing. Any software of sufficient complexity has bugs, and fixing the bugs or "improving behaviour" can bring about unintended new bugs.

It's not that people are deluded to expect totally bug free software, it's just that they don't know what's involved. It is akin to wishing for a world full of peace, love and free beer. It ain't going to happen.

[BletchleyGeek]

Okay, sburke I'm not coming here to pick up fights, and probably my Spanish upbringing might sound very impolite in settings where the exquisite Anglo-Saxon etiquette is the norm. So bear with me if I sound too harsh (or frank).

That's totally understandable and I understand it. My observations regarding testing come from my background on stuff like developing and maintaining a speech recognition and synthesis systems (one that actually worked and was bought off by a big company which shall remain nameless), computer vision systems (reconstructing 3d figures from 2d pictures), and agent-based simulation and artificial intelligence systems. In all those works the team I worked in made good use of 1) actual people testing things and 2) automated testing over benchmarks along with simplistic yet effective programs using heuristics to detect when things go off.

This thing of the bad plate / niggardly quality rating assignment is the kind of thing one spots better with automated testing than with human-driven testing. Why? Just imagine you're having a quite massive database of vehicles, each of them characterized by a substantial number of parameters. For instance, let's say you have a parameter that prescribes how probable is a vehicle to catch fire when its armour is penetrated.

If that vehicle is a Sherman, indeed, if when assigning this 'ronson' parameter one was too generous (or did a typo when entering the data) it will be quickly spotted since 1) Shermans are all over the place in scenarios and 2) everybody knows Shermans catch fire easily.

But what about the Sdkfz 253? Or the Panzer IIIN? Or an early Churchill model? How often does one of those appear in an scenario? How many people have a good sense about what to expect from those vehicles when penetrated?

A quite easy heuristic to implement (if the engine allows it, it can take considerable work to retool the engine so it can be harnessed by benchmarks) is just to have a little script that runs a test program that loads the data and generates different penetration events for the vehicles in the database. Say you generate 100 penetration events and you get 0 brewing up results when you were expecting a 50%-50% chance. Then writing to a log "Vehicle X seems to be hard to catch fire, check Y", is quite easy and goes a long way to iron out the problems that complex simulations with many parameters entail.

[sburke]

Okay, sburke I'm not coming here to pick up fights, and probably my Spanish upbringing might sound very impolite in settings where the exquisite Anglo-Saxon etiquette is the norm. So bear with me if I sound too harsh (or frank).

Don't sweat it, I have a fairly thick skin. Doesn't mean I can't be an a**hole. I'd have never guessed a Spanish upbringing from that handle though.

Your post was far more into technical aspects of how to generate high capacity results testing. I am in telecom and I get that. We do tests to verify some stuff, but to verify conditions under load we need a traffic/call generator to see a system response. Only BF could answer your questions, I am not qualified as I don't see what they do at that level. My response was to completely different posts. Sorry for the confusion.

[sfhand]

...4 days is an incredibly short window to develop, test and release a patch...

And here comes sburke, pissing on everyone's hopes of a patch before the weekend...

[sburke]

And here comes sburke, pissing on everyone's hopes of a patch before the weekend...

rotflmao Hold on there, I said it was short, but they did do it.

[ASL Veteran]

This thing of the bad plate / niggardly quality rating assignment is the kind of thing one spots better with automated testing than with human-driven testing.

Maybe you missed it (in spite of your extensive experience with noticing things), but it was explained that the plate quality rating was deliberately set to low quality by the person who codes it. It was intentionally set that way. In other words, that was not a bug. It was intentional.

When someone complained about it, the decision was made that it was probably easier to just alter the plate quality rating than to argue back and forth with forum members about whether it was the correct rating or not since the charge was being made that the game was inaccurate with regards to penetration of that particular plate. Once that happens, then the Armor Plate Pope calls for a crusade and the Jpz IV Plate Templars fire up a new thread every other week demanding a fix to the broken plate because they can't play the game anymore. The incorrect plate value has broken the game to the point where it isn't even worth playing the game and they would rather spend all their time on various gaming forums complaining at the irrepairably broken armor penetration model in CMx2 unlike CMx1 which was handed down to Hercules by the hand of Zeus for the masses to enjoy in all it's perfection. The only way to correct the matter is to pressure BFC with the blunt force of hundreds of repeated forum posts until BFC finally breaks and a proper solution is brought forth to please the masses.

As demonstrated by the continued discussion of the Jpz IV plate, even the immediate adjustment of the plate value apparently was insufficient to distract the Jpz IV Plate Templars who apparently are continuing on in their crusade in spite of the fact that they have already won an easy victory.

[Vanir Ausf B]

When someone complained about it, the decision was made that it was probably easier to just alter the plate quality rating than to argue back and forth with forum members about whether it was the correct rating or not

Of course. That makes perfect sense given that the person who makes those decisions does not post on the forum. But certainly if he did there would have been a rousing flame war on subject, give that not one person ever claimed that the armor quality rating of the Jpz IV was too low.

As demonstrated by the continued discussion of the Jpz IV plate, even the immediate adjustment of the plate value apparently was insufficient to distract the Jpz IV Plate Templars who apparently are continuing on in their crusade in spite of the fact that they have already won an easy victory.

I probably should not let inconvenient facts get in the way of a good rant, but as a Plate Templar I feel obligated to point out that the continued discussion in that thread has had almost nothing to do with the Jpz IV, as anyone who had bother to read it would know.

[Battlefront.com]

Would now be the right time to point out that the JpzIV plate armor "bug" was in all 3 CMx1, CMBN v1.0, and CMFI and nobody mentioned it until nearly 13 years after the first time it was played with?

Or would that create a fuss?

Steve

[Battlefront.com]

We've had discussions about automated testing before. Even if we spent equal amount of time on such a testing scheme (which still would be imperfect) as we do making the game, it wouldn't catch much of anything we don't already catch.

The system we have works just fine in the real world we live in. Since that's the only one that matters, we're not making any changes. Unless, as I offered a few pages ago, you guys want us to do less and take longer to release. THAT would result in fewer bugs.

Steve

[MikeyD]

Huh, and all this time I thought people were arguing in in favor of getting a weaker Panther lower hull plate to match the JpzIV!

People do know, don't they, that different vehicle types have always been assigned differing quality armor in the game. Some armor is soft, some spalls easily, some is just plain crappy, some is excellent. Sherman types can vary widely, especially in CMFI with its early-build Shermans. I'd expect armor quality to play an even larger role in Eastern front and Bulge battles so you'd better get used to the concept.

[Vanir Ausf B]

Would now be the right time to point out that the JpzIV plate armor "bug" was in all 3 CMx1, CMBN v1.0, and CMFI and nobody mentioned it until nearly 13 years after the first time it was played with?

Or would that create a fuss?

Steve

There seems to be a little confusion about that thread. No one ever claimed the armor quality rating for the Jpz IV was wrong. Because initially no one knew it was rated at something less that 100%. So of course the assumption was that it must be a thickness or angle mistake, which test results strongly indicated, had the armor actually been rated at 100%. Which would have been a bug. For all I know, the armor quality of the Jpz IV may be correct since I have no idea where Charles got his information from an I have no information that explicitly says it's wrong.

[Battlefront.com]

The point I made in the other thread is that knowing the numbers didn't help anybody "debug" the problem for the 4 years of active development of the three games with that data. People saw the numbers and didn't question the results, therefore didn't question the numbers. And yes, armor quality was displayed along with the thickness, angle, armor type, and other factors.

The fact that someone figured there might be a problem WITHOUT data proves that the game has sufficient information to figure out if something is right or wrong. It's not necessary for you guys to figure out the specifics of what a problem is, or it's possible solution, only to note where there's an issue. We can handle it from there, whether it be a mouse cursor that isn't working right, a graphic coming up in the wrong spot, or a shot going through some armor that probably shouldn't.

Steve

[Vanir Ausf B]

People do know, don't they, that different vehicle types have always been assigned differing quality armor in the game. Some armor is soft, some spalls easily, some is just plain crappy, some is excellent. Sherman types can vary widely, especially in CMFI with its early-build Shermans. I'd expect armor quality to play an even larger role in Eastern front and Bulge battles so you'd better get used to the concept.

Yes, people do know that different vehicle types have always been assigned differing quality armor in the game.

The real questions are: do "people" know that there is no in-game indication of this? Do "people" know that for other units with armor quality reductions such as the Sherman, Panther and Tiger there is specific mention made of it in the manual(s)? Do "people" know that there is no such mention of armor quality issues in any of the manual entries for Jpz IV? I get the feeling some "people" know very little about which they speak.

[Vanir Ausf B]

The fact that someone figured there might be a problem WITHOUT data proves that the game has sufficient information to figure out if something is right or wrong. It's not necessary for you guys to figure out the specifics of what a problem is, or it's possible solution, only to note where there's an issue. We can handle it from there, whether it be a mouse cursor that isn't working right, a graphic coming up in the wrong spot, or a shot going through some armor that probably shouldn't.

Yep. I agree For the most part the ballistics modeling in CMx2 is superb. I think that's what makes people take notice of the occasional oddity.

[JasonC]

Judging by ASL-Vet's post, I'd have to say "yes" to the original thread title. When attacking the loyal fan base and defensively bad mouthing the glory days of CM1 is a true believer's first instinct, I'd say Fonzi has gone airborne...

[Battlefront.com]

Yes, but if Pinky Tuscadero is saying that Fonzi can't jump over those cars because his jacket isn't buttoned the same way as yesterday, then what should happen? Have Fonzi put his head down and go to his "office" to hit the wall? Or have him give that look he is well known for?

Unfortunately there are still plenty of CMx1 fans that have rose colored glasses on. When someone like that comes here and tries to pour cold water on something they don't understand or aren't fairly (objectively) analyzing, what is one to do? Pretty hard to challenge an assertion without addressing the root cause of the reason for it.

I can say from my perspective it's rather annoying to have people routinely trying to figure out how well we're doing as a company and as a game system, then concluding that we're "dying". I don't know what you do for a living JasonC, but I don't think you'd be happy to have your job performance be scrutinized (negatively) with a metric that was completely irrelevant or incorrectly applied.

Steve