Jump to content

Update on Engine 4 patches


Recommended Posts

One of the companies that we got some COTS radiation analysis software from - small company - called me their Charlie tester. Just when they thought they'd covered everything I'd stress the code a little farther and find something no one thought of. To be fair to them, I was finding new ways to use their code on unique problems, so they aren't really to blame. 🙂

Link to comment
Share on other sites

While I do know and have experience with the value of all sorts of testing among which automated regression tests, unit testing, CI/CD and the likes, they don't come for free, are more complex processes and aren't a magic tool to always fix everything. Even apart from the often required change management among developers 😉

I think there is a big difference with regards to requirements for testing depending on the type of products or service a company offers. If your application is a live production service, of which important client business processes are dependent, you'd have a different tolerance for bugs compared to a computer game, let alone a niche wargame. And probably a different resource budget, strategy and policies regarding (automated) acceptance testing of changes/updates. ISO certificates, yawn, etc.

Of course I'd also like CMx bug free, but I don't see how a testing 'plan' would change all that. To plan, and to make plans, is very important. They are however just one small step in the whole thing. Worked with quite a number of 'project managers' that think that just making a plan or a report for anything just magically fixes the whole universe and allows you to tell 'when it is done'.
Like Steve says, if BFC decides to increase their focus on testing this effort needs to come out some other basket and so the next release will probably have fewer bugs but less content or is later than it would be without the extra testing effort. If it would be magically possible to hire a tester and have she/he being productive without significant extra time overhead for the team, it would mean each next game/module/pack is more expensive (and there would still be bugs, although hopefully less). 

The wait for the 4.0 patch was very long indeed, but that has been been discussed over and over. Personally I think that the communicated plan to do smaller patches a bit more often is a good plan. CMSF2 got a patch within a month. And good to know the 4.0 patch is here 'real soon' :D !

Edited by Lethaface
Link to comment
Share on other sites

yeah this is a more complex discussion that has a lot of dependencies.  Automating as many functions as possible without creating simply another burden is a balancing act.  It's great when it works, but as lethaface noted it doesn't come for free.  If a process that is designed to for example catch stuff in QBs you'd need to set it up to run a multitude of force selection options that are unique to each title.  Then you have to decide how granular it gets.  For CMSF2 that also means having it run against a rapidly evolving ToE.  Mortars for example were a pain in the butt in this title as on board didn't even exist in CMSF1.  Force selection options for WW2 titles don't mean the same as modern.  Infantry only for example is a questionable selection in an era where mechanized/motorized resources are everywhere.

TOE alone is a huge overhead and there are a few individuals who live and breathe that stuff (I am most definitely not one of them).  That isn't something you can automate as it is mostly a research effort.

Undoubtedly there may be some areas that this kind of effort could produce results, but finding where those areas are where the effort is worth the result is tough.  Even something that might be considered relatively easy like taking the map and unit data to create the basics for the tactical maps for scenarios is probably not worth the effort involved.

I don't think the delay for the 4.0 patch would be helped by this type of thing.  That is behavioral testing and no AI program is gonna tell you whether or not you've reached some magical state to make everyone happy.  There is simply no way around that being a compromise discussion with BF about what type of behavior would be considered optimal and what kind of behavior can actually be programmed and then testing it repeatedly to see if the result is acceptable.  Unfortunately there are limitations to the possible and I suspect we will never reach a state where everyone is happy.

Lastly the process of creating scenarios runs parallel to game development.  Occasionally items get baked into a scenario that is created early in the development process.  This has come up a few times relative to CMBN with some very specific building and bridge issues in scenarios.

While I agree that smaller patches a bit more often is a good plan, it also has it's weaknesses as each patch still requires testing effort and depending on what is being patched (AI behavior) small patches don't mean small effort.  What may be a better plan is bucketizing the items by effort and testing involved and then allocating them based on how much testing is required. Size then becomes a variable that is determined by effort.

Edited by sburke
Link to comment
Share on other sites

Another example of something that needed a testing plan a bit more than a $60 game... from the New York Times:

https://www.nytimes.com/interactive/2018/12/26/world/asia/lion-air-crash-12-minutes.html

Seems Boeing should have tested their automated system for the possibility of receiving conflicting information and multiple Human attempts at an override.  Also would be nice if the pilots had been told what to do.

My point, again, is that the real world spits in the face of perfection.  It's fine to point out bugs in our games.  It's fine to expect us to fix them.  But expecting there's some magic wand out there which can waive away bugs at all, not to mention doing so without negative ramifications, is not reasonable.

As for the frequency of patches in the future.  We hope to not get ourselves into such a daunting task of trying to patch 5 games at one time.  Instead we're planning on patching games on a more one-at-a-time basis.  The downside of that strategy is game features will be out of synch for a while, to the extent the bugs fixed are related to Game Engine elements.  This is what we had intended on avoiding with the Engine 4 patches, but I don't think anybody is really happy with how that worked out.

Steve

Link to comment
Share on other sites

5 minutes ago, Battlefront.com said:

My point, again, is that the real world spits in the face of perfection.  It's fine to point out bugs in our games.  It's fine to expect us to fix them.  But expecting there's some magic wand out there which can waive away bugs at all, not to mention doing so without negative ramifications, is not reasonable.

I think maybe this was directed a bit at my previous comment. After reading what I wrote again, I admit it was too blunt and not entirely reasonable. Bugs and mistakes happen everywhere in any production, and I didn't mean to come across as that cranky.

Also, I appreciated the many news updates lately.

Link to comment
Share on other sites

15 hours ago, coachjohn said:

Could be worse. Look at Command ops by panther games. Every time they have a major patch / upgrade you have to repay for each of your modules / DLCs to be upgraded. About 50% of original price.

 

Or maybe paradox that has milked everything from their customers they can.

Command Ops gives you the base game for FREE, including several great scenarios. The DLC is worth every penny. The small but active community also produces nice scenarios (about Caen for instance) and mods for free. It's a great game with lots of potential (perhaps Eastern front in the future). I also don't get what you say about repaying. Doesn't make sense to me.

Edited by Aragorn2002
Link to comment
Share on other sites

6 hours ago, Aragorn2002 said:

Command Ops gives you the base game for FREE, including several great scenarios. The DLC is worth every penny. The small but active community also produces nice scenarios (about Caen for instance) and mods for free. It's a great game with lots of potential (perhaps Eastern front in the future). I also don't get what you say about repaying. Doesn't make sense to me.

It may be old data but on their site it talks about the updates and modules. If the core game is updated you have to pay to get your modules updated. They were talking about possibly doing 2 updates a year so if you want your dlc to take advantage of the core update you have to pay. How that works out financially in real life terms isn’t clear to me but one thing I did see is they did not get acct info when moving to lock and load so I’d have to start from scratch. 

Interestingly when they presented it they talked about the alternative of having to maintain separate games. BF has gone in a different direction but you can still see they have both faced similar challenges. This is also a path that has been discussed for BF on this forum i.e. doing a base engine install and then having all the modules applied to that engine. It will be interesting to see if this pans out for command ops. Hope so for them as it is a unique game that is worth keeping around. I am not much interested in real time stuff but theirs is an exception. 

Link to comment
Share on other sites

On 1/4/2019 at 2:56 AM, Aragorn2002 said:

Command Ops gives you the base game for FREE, including several great scenarios. The DLC is worth every penny. The small but active community also produces nice scenarios (about Caen for instance) and mods for free. It's a great game with lots of potential (perhaps Eastern front in the future). I also don't get what you say about repaying. Doesn't make sense to me.

Yes, but the CO2 long awaited Volume 8 "Patton's Charge/Bradley at Bay" is 18 months past its published release date.  The devs seem to encounter endless problems with the upgrade and no new release date is in sight.  It seems they need a great deal of time to insure release a quality product as well.

Link to comment
Share on other sites

On 1/4/2019 at 10:56 AM, sburke said:

Interestingly when they presented it they talked about the alternative of having to maintain separate games. BF has gone in a different direction but you can still see they have both faced similar challenges.

Back when we started thinking about expanding the original Shock Force we had several choices to make:

1.  Do we add onto one game or release stand alone games?

2.  If we went with the addons for one a single game, can customers purchase whatever addons they want or do they have to purchase everything sequentially?

3.  Do improvements in the game engine come with the addon for free or is it a separate purchase?

These were big questions and we thought long and hard from the customer, engineering, and marketing (business side).  What we initially came up with was new content would be addons at a reasonable cost which were not required in any particular order.  We also decided that improvements to the game would be minimal and therefore did not justify charging customers for them.

As things moved along we found two co-dependent questions in need of answers:

4.  Do we allow a particular game (Family) to gain new significant features available to others, or do we force people to repurchase the whole thing all over again, or do we let it go stagnant and die of old age?

5.  If we do provide a path to adding new game features to existing games, do we also charge for addon content to be upgraded or should it come along for the ride at no extra charge?

After seeing the long term interest in CMSF and CMBN we decided we should offer what turned into Upgrades.  Customers do not have to repurchase the whole game all over again, nor do they have to suffer "feature envy" as newer releases come out with major improvements.  We also thought it was really bad form to require people to repurchase content they already owned, even if we had expenses in updating that content (which we do, BTW).

Shock Force was in a tough spot because far too much had changed to provide a one-size-fits-all $10 Upgrade like we had for the other games.  But we didn't want to make people pay for everything all over again, despite being justified in doing so (2 years of development ain't cheap, after all!).  Which is why we created the inexpensive special Upgrade options that are now available.

Based on our 10 yeas of experience doing it this way, it's our feeling that we got the balance of elements right.  Or at least right enough ;)

Steve

Link to comment
Share on other sites

I for one have no complaints at all with how you have decided to do the upgrades on all the games and how you have priced the add-ons and upgrades to the engine.

I also am impressed with CMSF 2 in that you made it work being able to load old CMSF 1 scenarios.

When I read complaints about how you are pricing and marketing games, I generally see it appears to be a customer who either is not someone that is going to spend much time with the product or its someone that thinks the world owes them things for free. ( Both of which have come from a gaming industry that teaches them that those two principles can be somewhat true).  So I have no feelings of remorse for them when they find out this company does not work on such concepts.

I also have found that I have never regretted a purchase of your games  in that I likely get about 800 hours a year of gaming entertainment with you products.

Where as, there is hardly any other game that has managed a 100 hours of my time in the whole time of ownership and many a game that has not given me 10 hours of time before I never want to see it again.

Keep up the good work and live long and prosper

 

Link to comment
Share on other sites

"Your most unhappy customers are your greatest source of learning."
Bill Gates

Sometimes customers have the greatest ideas... and sometimes not but either way a good business learns from them. Battlefront's track record speaks volumes on your ability to learn, adapt and create entertaining, interesting games that are a great value.

Shock Force was in a tough spot because far too much had changed to provide a one-size-fits-all $10 Upgrade like we had for the other games.  But we didn't want to make people pay for everything all over again, despite being justified in doing so (2 years of development ain't cheap, after all!).  Which is why we created the inexpensive special Upgrade options that are now available.

Based on our 10 yeas of experience doing it this way, it's our feeling that we got the balance of elements right.  Or at least right enough ;)

Steve

CMSF2 is a clear example getting it right.

Buzz

Link to comment
Share on other sites

9 hours ago, slysniper said:

I also have found that I have never regretted a purchase of your games  in that I likely get about 800 hours a year of gaming entertainment with you products.

Where as, there is hardly any other game that has managed a 100 hours of my time in the whole time of ownership and many a game that has not given me 10 hours of time before I never want to see it again.

Keep up the good work and live long and prosper 

I regularly buy games for my boys for their PS4 and XBOX and  it's  50 € each time. I remember one that the elder finished in 4 hours!
when I discovered CMBN, I did not have any games on my PC and I had to buy one with enough power and a graphics card (talking to Madame about the interest of such a computer for the studies of children you can lift mountains ...)
Then I buy the CMBN game. like i do not have enough computer flow I buy it in disc with customs fees
All combined, game and PC , I paid a little more than 1000 €
Well, I do not regret!
The number of hours passed since but especially the pleasure of the game justifies more than largely such expense
I happen to buy a good bottle of wine at 100 or 200 € which ends in 1 hour I do not regret either, everything is a matter of choice

Link to comment
Share on other sites

12 hours ago, Battlefront.com said:

Based on our 10 yeas of experience doing it this way, it's our feeling that we got the balance of elements right.  Or at least right enough

You guys have made some very smart decisions over the years.  It's a tribute that BF is one of the very few hardcore wargame sim developers still in business.  And the others are doing much less ambitions/interesting/compelling products.  Have always internally calculated the value of each CM game family to be in the approx $300 range and with all the modules etc. available that's about right.

My concern would be that at the current rate of production it will take a very long time to get to the early war.  In the meanwhile, the play experience difference between a 1944 era game and a 1943 era game (ie: a one year difference) is not that appreciable. 

One of the great features of CM1 was the ability to fight Barbarossa one day and a 1945 scenario the next.  It kept things fresh.  Of course we all understand that while CM1 was unbelievably fabulous value for customers it is not a commercially viable biz model for BF.  The dream has been that one day we would have 1941 to 1945 products all available at the same time.  If the CM2 engine is still viable 5+ years from now maybe we'll get there.  But, it could take a lot longer than 5 years.

Currently, CM2 is effectively 2007 era graphics and despite all the wonderful engine improvements (to me at least) the system offers essentially the same gameplay experience.  Now that the CM2 system is 12 years old one has to wonder how many more years can it keep satisfying customers.  We all agree that graphics is not what motivates our interest in the CM system.  But, at the same time not many of us are playing classic games like "pong" or the 1990's simulations.  

Am certain that BF has given much thought as to what the next evolution (CM3) will be and when it may be developed.  So, there is nothing in what I have said that will be news to you guys.  Just wanted to congratulate you on the success of the system but also voice my concern that things have to evolve well before your customer base gets bored.

 

Link to comment
Share on other sites

3 hours ago, Buzz said:

"Your most unhappy customers are your greatest source of learning."
Bill Gates

Very true but a bit one sided. I guess the trouble I have is the "most" part. Looking at the patterns of unhappy customers is how you find areas that need attention. At the company I work at we have used that feedback to update the way we document features, reworked whole networking messaging because of configuration confusion and remove whole features in favor of using third party systems that were in common use already by our customers.

However the 'most' unhappy customers are actually just pissed off, frequently unreasonable jerks. Sorry like all things human there are jackasses out there and you run into them. Those angry unreasonable people are not a source of learning - unless you are talking about learning to stay polite in the face of rudeness :) Analyzing thier problem along with all other incidents *is* the source of learning.

A long way to say being polite and expressing your problem *is* helpful and contributes to making a better product. Being an entitled jackass contributes nothing more, while at the same time making you difficult to deal with and harming those that have to deal with you.

Edited by IanL
Grammar and tyoo
Link to comment
Share on other sites

1 hour ago, IanL said:

Very true but a bit one sided. I guess the trouble I have is the "most" part. Looking at the patterns of unhappy customers is how you find areas that need attention. At the company I work at we have used that feedback to update the way we document features, reworked whole networking messaging because of configuration confusion and remove whole features in favor of using third party systems that were in common use already by our customers.

However the 'most' unhappy customers are actually just pissed off, frequently unreasonable jerks. Sorry like all things human there are jackasses out there and you run into them. Those angry unreasonable people are not a source of learning - unless you are talking about learning to stay polite in the face of rudeness :) Analyzing thier problem along with all other incidents *is* the source of learning.

A long way to say being polite and expressing your problem *is* helpful and contributes to making a better product. Being an entitled jackass contributes nothing more, while at the same time making you difficult to deal with and harming those that have to deal with you.

yeah +1 on that.  There is a difference between my most unhappy customer and my most reasonably unhappy customer.  I am so sick of folks who have no knowledge and somehow think everything can be magically integrated and to think for them.  Sure I can do that, how much money you have to spend on it... oh wait none?  Yeah I'll have that done as soon as the money is in my acct.  I am soooo looking forward to retirement when the only person I have to worry about pleasing is my wife who is reasonable, nice and appreciative.

Link to comment
Share on other sites

13 hours ago, slysniper said:

I for one have no complaints at all with how you have decided to do the upgrades on all the games and how you have priced the add-ons and upgrades to the engine.

I also am impressed with CMSF 2 in that you made it work being able to load old CMSF 1 scenarios.

When I read complaints about how you are pricing and marketing games, I generally see it appears to be a customer who either is not someone that is going to spend much time with the product or its someone that thinks the world owes them things for free. ( Both of which have come from a gaming industry that teaches them that those two principles can be somewhat true).  So I have no feelings of remorse for them when they find out this company does not work on such concepts.

I also have found that I have never regretted a purchase of your games  in that I likely get about 800 hours a year of gaming entertainment with you products.

Where as, there is hardly any other game that has managed a 100 hours of my time in the whole time of ownership and many a game that has not given me 10 hours of time before I never want to see it again.

Keep up the good work and live long and prosper

 

+1

Link to comment
Share on other sites

11 hours ago, Erwin said:

But, at the same time not many of us are playing classic games like "pong" or the 1990's simulations.  

Now, that just wrong, Erwin...I still have my 'Pong', Atari, and Intellivision Console Games in the closet for Nostalgia reasons :-) 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...