Any plans for 1.12?

[Wodin]

@BF, See post 15..I said I expect version 2 bugs will be fixed with patches and you replied not necessarily...I was surprised as you by the response..maybe you miss read my question as version 1?

Obviously a misunderstanding..

[A Canadian Cat]

The first is in regards to the quantity of people being plagued by the various PBEM [crash dump], [5% hangup] etc.

How did this get passed testing?

Stuff always gets through testing, even for products with literally thousands of testers (Microsoft [fill in blank product] is a perfect example).

For us, it's the downside of having a game with so much going on and so many different ways to play. It is impractical to test for every single possibility every single time we get a new build to play with. For all I know there was no PBEM problem for the first 3 months we were testing and it was only introduced the last build.

True, stuff always gets through testing. It is unavoidable - at least if you plan to ship product

I do have a suggestion though. This is something that we have been doing a little at a time on the product I work on - automated testing. Our portion of the product is primarily UI so this has been particularly challenging and I am certainly not going to claim perfection but it has helped.

In your case you already have an AI capable of fighting against a human. Some tweaks and you could have the AI fight itself. [aside: might even be a useful feature for scenario testing]. Add a command line way to start a new game and you could automate the testing of scenarios. Now you can be sure that scenarios with barbed wire are always going to work.

Next, add a way to define a Quick battle start up from the command line. [aside: might be a useful feature for those of us making tools to initiate a battle from an operational layer]. Now you can automate starting and running RT quick battles AI vs AI. Now you can be sure that QBs can be started and run to completion at least for RT play.

Next add a way to start a Quick battle .ema file from the command line. [aside: might be a useful feature for those of us making tools to manage turns for PBEM]. Add a way to auto save the new .ema turn file at the end. Then wrap all that with little scripting to start .ema files run a turn and copy files around and start the next turn. Couple that with a modified way to pick QB forces and bingo now you know that customers can start, choose forces and run a PBEM game to completion.

Slowly build your set of automated tests when you run into issues. It all helps. None of this replaces any other testing because there is nothing here to be sure that units are behaving as expected, that models look right etc. etc.

[aside: yes I know some of the things that you would need to add for automated testing have been asked for by people like oh say me . I am just showing how those thing could be useful to you too]

[poesel]

Very well put Ian! +1 to that.

[Battlefront.com]

That's pretty boggling, Steve. Are you saying there was no code reuse between CMx2 v1 and v2? That every function was rewritten?

Of course not. I'm saying the ability to port things over is conditional on what specifically is being discussed. The more isolated and simplistic the change, the more likely it can be ported backwards with significant difficulty. But what is isolated and simplistic to the lay person is not necessarily that way to the programmer.

The mortar targeting thing is just a state that's not being reset when it should be.

You have no basis for making such a statement. It could be that simple, or it could be extremely involved. I don't know where on the spectrum it falls, but I *do* know it wasn't even close to being as simple as you think it was. The variables get set by AI functions which obviously hadn't taken certain cases into account.

Personally, the only x that I care enough about to fret is the mortar targeting thing.

I assume the fix will be in the v1.12 patch because I was part of the discussion that lead to it being fixed. It seems isolated enough to be portable. But I am also not a programmer so I can't say with 100% certainty. I know it's on the list, but we're working on v2.01 at the moment and haven't looked at it specifically yet.

They weren't snide, they were hyperbolic. And they were in response to someone stating an opinion that you weren't doing a 1.12. I asked those questions in the hope that the answer would be, materially, what you answered.

You had a range of options how to respond even within a chosen form of hyperbole. The choice made reflects the underlying attitude you wished to express. Given the previous posts you made within a few hours I don't think my assessment of your attitude was off the mark.

I have to apologise for that particularly hubristic and solipsistic point. I can offer in mitigation only that it was late, and I wanted to answer before I hit the sack, so I rushed.

Now we're getting somewhere.

Did you read my initial post? That was, I believe, entirely without any potential for interpretation as rude or disrespectful.

First post was fine. Second post was also fine. Third post is where you started to go wrong from my perspective.

The point I'm trying to make, I think, is:

1) There is a bug

2) It's fixed in the paid-for upgrade

Correct.

3) It's not fixed in the free patch

The v1.11 patch was completed before the mortar bug deployment issue was fixed. So of course it wasn't in v1.11.

4) This generates the perception that you are charging for bug fixes.

I know that perception can become reality, but it doesn't make it reality. Given our track record we should be given the benefit of the doubt. Especially when our record is compared against other game companies, or for that matter software companies in general.

That prception is harmful to you, even when it's only some bugs. I have, here and in other places, defended BFC from broad sweep "Charging for patches" accusations. I don't believe you ever intend to stop giving free patches for bugs. My agenda, which should be perfectly obvious, is to get the mortar bug fixed in CMBN v1 so I can honestly say that I believe you'll fix all the serious bugs before dropping support for a line.

We will do our best, but I am sure we will drop support for CMBN v1.x with some issues remaining that someone will call "serious". I can say that because I still see people sharpening axes over "serious" issues present in CMBO/CMBB/CMAK. Just be glad you didn't pay $10,000 for it and then are expected to drop a couple thousand for an equally bug ridden upgrade. You 3D artists out there might have a guess as to what product I'm talking about.

Steve

[Battlefront.com]

True, stuff always gets through testing. It is unavoidable - at least if you plan to ship product

Yup. And even if we shipped a game with 0% problems today, tomorrow a driver or OS update could change all that. Too often it does

The automated testing scheme only works for specific circumstances, namely crashes/lockups. Circumstances which are rarely missed during testing and are generally easy to fix once identified. I'm not sure the effort to make an automated testing system is worth it because it would have to be programmed and then extensively tested to make sure the scripting itself isn't causing bugs. This takes man hours of labor for something that probably will be caught with almost no effort during routine testing.

We have far more problems with general gameplay variables not doing what we expect them to do. And for that, unfortunately, there is no automated solution possible.

Steve

[womble]

You have no basis for making such a statement.

I don't have to be wrong all the time, Steve. I do have a basis: analytic logic. It was a very broad statement, and your answer even shows that it was broadly correct:

I said: "It's a state"

which is represented in the code by

...The variables...

I said "...that's not being reset when it should be..."

which seems congruent with:

...set by AI functions which obviously hadn't taken certain cases into account...

Whatever. The important thing is it's being looked at and may be in 1.12. I'd be pretty surprised if there were so many other dependecies on this that it wasn't portable.

So. What's the plan for bringing the v2.0bis features (unit patches and such scrumptiousness) into FI?

[dungeontiger]

The automated testing scheme only works for specific circumstances, namely crashes/lockups. Circumstances which are rarely missed during testing and are generally easy to fix once identified. I'm not sure the effort to make an automated testing system is worth it because it would have to be programmed and then extensively tested to make sure the scripting itself isn't causing bugs. This takes man hours of labor for something that probably will be caught with almost no effort during routine testing.

We have far more problems with general gameplay variables not doing what we expect them to do. And for that, unfortunately, there is no automated solution possible.

Steve

Steve,

I was enjoying this tread until you add the above statements. Now you are really scaring me. Automated tests are only good for crashes? Are you serious?

Automated tests and unit tests do not general test for crashes. They specifically test for expected output. You must know what is expected as output from your algorithms. It must be deterministic or it would not work.

Sure, it is unreasonable to say build an automated test that runs any case and identifies anything you did not expect. That's too generic.

However, it is totally reasonable to expect you to create isolated situations using part of the code and give it input and verify you get the output you expect.

Consider the wandering crewman problem. You have enough logged turns with this issue to reproduce the problem and identify what caused it. Now you can create a test that takes only the minimum to replicate the problem. It could be a turn with a single crew served weapon in a specific terrain. The input is a new command to the crew. The expected output is the micro commands that your engine gives to each soldier. Obviously one member is receiving a bad micro order (I'm speculating on how your engine works of course). Your unit test would verify that they did not receive this bad order. Once you automate it, you can guarantee that this bug will never come back (be reintroduced) because you are testing for it each night and before each check in.

Automated testing and unit tests don't generally prevent new bugs in new code. There primary job is to prevent old bugs from being reintroduced and ensuring that code that worked before checking in work continues to work as expected after the check in. How else can you be sure that you did not break something when you check in code?

Automated testing and unit testing is one critical tenant of any serious software project. Not having automated testing and unit testing is like not using source control or having nightly builds (if you don't have that please DON'T tell me, I'm scared enough as it is now).

Yes, it definitely requires an investment to implement automated and unit testing. In my young and foolish years (now I'm just foolish) I felt such work was a waste of time. "I'm smart enough to keep everything in my head and I test like a customer so why should I waste time on this?" Now I know better. I cannot tell you how many dividends have been reaped because of the time and effort I put in to ensure my code was properly unit tested via automation.

In all seriousness, I'm sure your engineer(s) is much smarter than I am. Seriously. But that does not mean you don't require this framework. If anything, once the investment has been made, it will free up the brilliant mind(s) you have to safely implement the new features we are all clamoring for.

Please, I don't want to sound negative or ungrateful. Your pricing model is great and your quality is generally higher than most products. I have no problems to support you. It just sounds like you could be even better and stronger with a little more software engineering principles.

Stephen.

(boy, I sure feel a little slimy writing this. "Ye of no sin..." and all that. Please take it in the light of "here's some hard won experience that I think could make your wonderful product even better.)

[Redwolf]

I was rather hoping that the 1.x line and the 2.x line of development would be the same code, just with the new toys turned off.

That would be the only way to keep releasing 1.x patches without getting into an utter nightmare of porting fixes.

[sfhand]

Apparently lost amidst the wailing and gnashing of teeth:

... We also will certainly stop supporting v1.x when we release v3.0 standard sometime towards the end of 2013.

...

Steve

I'm still putting my money on this version, 3.0, being an unexpected addition to George RR Martin's current work, probably Bulge.

Isn't that vastly more interesting than taking shots at a small outfit trying to make the best wargame possible?

[A Canadian Cat]

The automated testing scheme only works for specific circumstances, namely crashes/lockups. Circumstances which are rarely missed during testing and are generally easy to fix once identified. I'm not sure the effort to make an automated testing system is worth it because it would have to be programmed and then extensively tested to make sure the scripting itself isn't causing bugs.

Automated testing is very good at catching crashes and lockups - true. The thing is you have had two instances that I recall that did make it out: 1) the barbed wire crash which took a long time before we saw a fix. 2) the quick battle 5% crash which was just introduced. I am sure that you would rather have not have released those and would like the time and mess they caused back.

This takes man hours of labor for something that probably will be caught with almost no effort during routine testing.

I cited two examples in your project that were missed during testing that would have been caught by automated testing. You guys are the only ones in a position to weigh the costs of building up an automated testing suite vs the cost to fix things later (both in terms of dev time and customer goodwill). Your, excellent, incremental approach to development can be applied here.

We have far more problems with general gameplay variables not doing what we expect them to do. And for that, unfortunately, there is no automated solution possible.

Very true - automated testing is no panacea here just one more more useful tool in the tool box.

Anyway you guys do have a good track record of servicing your games and we fans do appreciate that - well most of us do. I just saw and opportunity that, from my development experience, has helped us greatly and wanted to share my thoughts.

[fatehunter]

Steve,

I just want to get on here and write in full caps with asteriks between that there are some people here that are a bunch of whining etc.

I cannot believe how some people just belabour a point, in that they somehow think they have been screwed over through some kind of patch/upgrade issue that they disagree with. The internet is off course the perfect anonymous forum. I think I fired one of these people yesterday.

Obviously none of these people have ever actually tried to produce something for money and have absolutely no idea of the pareto prinicple. You must adhere to the pareto principle to get ahead in life, or anything. Spending time on the other 20% gets you ignored, bored, mal-adjusted, or just plain dead. Wonder how many people would play your games if you made the first one perfect but never produced another game for 10 years.

Anything interesting changes and is never perfect. That is why it changes. The past is never perfect, never will be and it cannot be changed or made better. Learn about your human nature and realize the the perfect little comfort zone you want to be in is the exact place that is holding you back from living life!

Buy the goddamn upgrade. It costs three coffees at Starbucks; can take you 10 km in a car (or 5 in a truck); a drink at a fancy bar; a big bag of nails or the upgrade.

Seriously the bag of nails is the best deal. They will work silently holding up your house for the next 50 yrs. Meanwhile the $10 dollar upgrade has already caused me to use up 40 hrs of my precious life!

Keep up the good ****.

[fatehunter]

Steve,

I just want to get on here and write in full caps with asteriks between that there are some people here that are a bunch of whining etc.

I cannot believe how some people just belabour a point, in that they somehow think they have been screwed over through some kind of patch/upgrade issue that they disagree with. The internet is off course the perfect anonymous forum. I think I fired one of these people yesterday.

Obviously none of these people have ever actually tried to produce something for money and have absolutely no idea of the pareto prinicple. You must adhere to the pareto principle to get ahead in life, or anything. Spending time on the other 20% gets you ignored, bored, mal-adjusted, or just plain dead. Wonder how many people would play your games if you made the first one perfect but never produced another game for 10 years.

Anything interesting changes and is never perfect. That is why it changes. The past is never perfect, never will be and it cannot be changed or made better. Learn about your human nature and realize the the perfect little comfort zone you want to be in is the exact place that is holding you back from living life!

Buy the goddamn upgrade. It costs three coffees at Starbucks; can take you 10 km in a car (or 5 in a truck); a drink at a fancy bar; a big bag of nails or the upgrade.

Seriously the bag of nails is the best deal. They will work silently holding up your house for the next 50 yrs. Meanwhile the $10 dollar upgrade has already caused me to use up 40 hrs of my precious life!

Keep up the good ****.

Failed to heed my own advice and never rant on the internet. This does nothing to encourage discussion. Apologies for losing my cool. Should have waited five minutes. Unfortunately had just read another few threads on the whole upgrade vs patch thing and lost it.

[mjkerner]

Anything interesting changes and is never perfect. That is why it changes. The past is never perfect, never will be and it cannot be changed or made better. Learn about your human nature and realize the the perfect little comfort zone you want to be in is the exact place that is holding you back from living

Excellent!

[mjkerner]

Anything interesting changes and is never perfect. That is why it changes. The past is never perfect, never will be and it cannot be changed or made better. Learn about your human nature and realize the the perfect little comfort zone you want to be in is the exact place that is holding you back from living

Excellent! Actually, earning that put me in my perfect comfort zone.

[Battlefront.com]

I was enjoying this tread until you add the above statements. Now you are really scaring me. Automated tests are only good for crashes? Are you serious?

Yup.

Automated tests and unit tests do not general test for crashes. They specifically test for expected output. You must know what is expected as output from your algorithms. It must be deterministic or it would not work.

In theory, with a simplistic and fairly mundane situation, that may be somewhat practical. But for something like CM? Absolutely impractical.

However, it is totally reasonable to expect you to create isolated situations using part of the code and give it input and verify you get the output you expect.

No, it's completely impractical. Therefore it is unreasonable. There is no such thing as a single input source, nor is there is there a single expected output source. It is exactly why CM is so wonderful rich, complex, and inherently unpredictable.

An example of this is we had a situation where tanks weren't spotting things correctly. All kinds of "input" were tested and the "output" sometimes came out correct, sometimes wildly incorrect. Turns out a single type of tank had the tank commander turned around backwards in the 3D program that created it. So for that one tank the commander wasn't looking out the front of the tank, but out the back. I struggle to even conceive of how difficult it would be to set up a test that, proactively, would have been rigorous to have found the problem any better than our testers did.

Consider the wandering crewman problem. You have enough logged turns with this issue to reproduce the problem and identify what caused it. Now you can create a test that takes only the minimum to replicate the problem.

Who has the time to create 10000 tests to adequately, and accurately, examine each and every element in the game? Certainly we do not. And even if it were possible to set up individual benchmark tests for each specific function in the game, how do you propose we set up tests to see how they interact with each other?

Automated testing and unit tests don't generally prevent new bugs in new code. There primary job is to prevent old bugs from being reintroduced and ensuring that code that worked before checking in work continues to work as expected after the check in. How else can you be sure that you did not break something when you check in code?

The only way possible... people play the game and report if things have gone wrong. When we make changes the testers are aware of those changes. They test to see if those changes had ill effects. When we monkey with something major we specifically ask the testers to stress test it in context of other elements.

Automated testing and unit testing is one critical tenant of any serious software project.

I have worked in the games industry for just about 20 years. I am unaware of any game that uses automated testing. It is as impractical technically as it is financially.

In all seriousness, I'm sure your engineer(s) is much smarter than I am. Seriously. But that does not mean you don't require this framework.

I've been making games for 20 years and never once had this in place. So yeah, I think I can seriously state that it means this sort of thing isn't required.

If anything, once the investment has been made, it will free up the brilliant mind(s) you have to safely implement the new features we are all clamoring for.

But it doesn't work that way. Coded tests would have themselves be debugged, altered as the game progresses, and generally maintained. It would probably double our workload, which would mean doing 1/2 as much for the game as we currently can. And still not have a bug free experience, because that's never going to happen unless we stop innovating within the game and adding new features.

Please, I don't want to sound negative or ungrateful. Your pricing model is great and your quality is generally higher than most products. I have no problems to support you. It just sounds like you could be even better and stronger with a little more software engineering principles.

Every game programmer I have ever met has said that they doubt their professors would last 1 month in the real world of game programming. Theory is always stressed in a practical environment, but games are probably the worst environment for sticking to text book practices. Games are more an expression of unbounded creativity. Very hard to engineer creativity even without economic pressures.

Steve

[Battlefront.com]

I was rather hoping that the 1.x line and the 2.x line of development would be the same code, just with the new toys turned off.

Too many fundamental changes to the code to have simple version toggles. There are toggles within each, of course, to keep things enabled/disabled for the various timeframes and fronts. For example, the code for thermal optics is present in the v2.0 codebase, but disabled.

That would be the only way to keep releasing 1.x patches without getting into an utter nightmare of porting fixes.

Yup, having two separate code bases does make back porting fixes more difficult. And since Charles is obviously not a dummy, he took this into consideration when making the decision between single or multiple code bases. I think it should be presumed that he had good reasons for doing that.

Automated testing is very good at catching crashes and lockups - true. The thing is you have had two instances that I recall that did make it out: 1) the barbed wire crash which took a long time before we saw a fix. 2) the quick battle 5% crash which was just introduced. I am sure that you would rather have not have released those and would like the time and mess they caused back.

Absolutely we would have rather not had them. But two serious problems in how many releases over how many years? How many things would we have had to code and maintain to have caught them over the same period of time? Honestly, I don't know. The only thing I know is Monday morning quarterbacking is the only thing that's easy to do If we bit the bullet and put in the time now, it could yield no result different than what we would have achieved without it. And since there's no way to know, for sure, I think I'd rather keep things the way they are now.

Anyway you guys do have a good track record of servicing your games and we fans do appreciate that - well most of us do. I just saw and opportunity that, from my development experience, has helped us greatly and wanted to share my thoughts.

Thanks! And for sure if I could snap my fingers and have a viable automated detection system set up to do even rudimentary automated tests I would definitely like to have it. But given our limited resources and massive demands put on us to add/change/fix stuff I'm guessing we're better off with the hit or miss approach in the big picture.

It's like the problems faced by modern militaries which spend inordinate amounts of money to keep each soldier safe from harm compared to the enemy. Last I read it costs $1,000,000 per year to field a soldier in Afghanistan for the US Army. For sure that one soldier can do things, and survive things, that soldiers even 20 years could couldn't have dreamed possible. Granted. But when the enemy can field endless amounts of soldiers at about 0.0002% the annual cost... well... "winning the battle but losing the war" is a saying that comes to mind.

Steve

[Battlefront.com]

I just want to get on here and write in full caps with asteriks between that there are some people here that are a bunch of whining etc.

While I agree with your overall point (i.e. if you REALLY care about the game AND the fix, then buy the Upgrade because it's a silly amount of money), we did tell people we would continue to support the previous version within reason. And that means we've entered into a contract and it's one we intend on honoring. If we should ever decide to alter the contract we'll have to change our marketing concept by phasing out the sale of the previous version games when we introduce a new one. This isn't good for anybody and so we intend on keeping up this system of dual support. Within reason.

Steve

[fatehunter]

While I agree with your overall point (i.e. if you REALLY care about the game AND the fix, then buy the Upgrade because it's a silly amount of money), we did tell people we would continue to support the previous version within reason. And that means we've entered into a contract and it's one we intend on honoring. If we should ever decide to alter the contract we'll have to change our marketing concept by phasing out the sale of the previous version games when we introduce a new one. This isn't good for anybody and so we intend on keeping up this system of dual support. Within reason.

Steve

Understood. and honourable.

Thanks

[JonS]

it costs $1,000,000 per year to field a soldier in Afghanistan for the US Army. For sure that one soldier can do things, and survive things, that soldiers even 20 years could couldn't have dreamed possible.

Fun fact:

At 22 years, the 1990/91 Gulf War is now much further back in time to us now than the end of the Vietnam War (which prompted the US military to take a long, hard look at itself, resulting in the battlefield dominance seen in Kuwait and Iraq) was to them then (~15 years).

[Lt Belenko]

....... This isn't good for anybody and so we intend on keeping up this system of dual support. Within reason.

Steve

Interesting choice of words "dual support". So are you saying when 3.0 comes out you will continue to support 2.0 and 3.0, letting 1.0 fall by the way side.

It really doesn't bother to me because I bought 2.0 the day it was released. But I'm just wondering???

[JonS]

Yes, that's exactly what he said.

Now, how much longer will be support v1.x? Longer than we would have under the old plan, that's for sure. But support will not be forever. Just like we don't support CMBO, CMBB, CMAK, CMSF, or CMA. No game is ever bug free, nor can it be, so that's an unreasonable expectation. At some point the games are considered "done" and we move on.

How many more patches will we do for CMBN v1.x? I don't know. We are planning on doing v1.12. We also will certainly stop supporting v1.x when we release v3.0 standard sometime towards the end of 2013.

[Battlefront.com]

Yup. Since we're talking about 1 year between major Upgrades the previous version will be, technically, 2 years out of date. That's an eon in game development terms.

Steve

[dungeontiger]

Every game programmer I have ever met has said that they doubt their professors would last 1 month in the real world of game programming. Theory is always stressed in a practical environment, but games are probably the worst environment for sticking to text book practices. Games are more an expression of unbounded creativity. Very hard to engineer creativity even without economic pressures.

Steve

This is most unfortunately. The right thing for me to do at this point would be to walk away from this thread. But when my credibility is attacked I'm compelled to respond.

I'm sorry you feel I'm a 'professor'. I'm a seasoned software engineer who has worked for some of the world's best software companies for 20 years. I have code that I wrote which runs on warships in some parts of the world and extensive amounts of business software that runs the biggest corporates in every major country of the world. Its likely you are indirectly using software I wrote without even realizing it.

I *think* I know a little about software engineering and clearly I am not a professor and I think have proven that I can survive in the real world and understand market pressures.

[calming down]

Like I said its unfortunately that you made such a rash assumption. Certainly you don't have to agree with anything posted here. I just felt that I wanted the record straight regarding my background.

I will not respond to the rest of your comments directly nor replying to this thread any longer since it clearly would not be productive.

Stephen.

[Mad Mike]

This is most unfortunately. The right thing for me to do at this point would be to walk away from this thread. But when my credibility is attacked I'm compelled to respond.

I'm sorry you feel I'm a 'professor'. I'm a seasoned software engineer who has worked for some of the world's best software companies for 20 years. I have code that I wrote which runs on warships in some parts of the world and extensive amounts of business software that runs the biggest corporates in every major country of the world. Its likely you are indirectly using software I wrote without even realizing it.

I *think* I know a little about software engineering and clearly I am not a professor and I think have proven that I can survive in the real world and understand market pressures.

[calming down]

Like I said its unfortunately that you made such a rash assumption. Certainly you don't have to agree with anything posted here. I just felt that I wanted the record straight regarding my background.

I will not respond to the rest of your comments directly nor replying to this thread any longer since it clearly would not be productive.

Stephen.

100% agree with you.

Steve doesn't know what he's talking about here, and it's actually embarassingly obvious and painful to read - for every software engineer with some experience who has ever worked on big projects in the real world.

Mind baffling response by Battlefront, really. :eek:

[Redwolf]

Too many fundamental changes to the code to have simple version toggles. There are toggles within each, of course, to keep things enabled/disabled for the various timeframes and fronts. For example, the code for thermal optics is present in the v2.0 codebase, but disabled.

Exactly.

So instead of having different codebases for 1.x and 2.x you use the 2.x codebase for both and just disable a couple new toys to warrant the price difference. That way there's no code porting nightmare.

I don't think there was anything in 1.x that is gone in 2.x.