PBEM turns not generating correctly for large battles

[stikkypixie]

I am currently playing a large CMBS game via PBEM. The turns generated are around 50 MB in size and both me and my opponent have had PBEM files generated which are significantly smaller than the expected 50 MB, rather they are only around 1 MB. We both have beefy machines and I checked the resource tab of my PC (Windows 10) while the file is being written and I still have plenty of resources left. Is this a known issue?

[Schrullenhaft]

I've heard of issues like this before, but I don't know what causes it specifically. It's best to go back to the player that first generated a 1MB file and have them redo their turn (that resulted in a 1MB PBEM file) and see if they generate an appropriate sized file.

I assume that even if you do nothing, the files should still be fairly large. Make sure that the directory that the PBEM files get written into is 'whitelisted/excepted' within whatever security/anti-virus software you use, just in case it has truncated/messed up the file as it was being written. Such interference, with this type of result, should be extremely rare, but it would be worth doing just in case.

[stikkypixie]

28 minutes ago, Schrullenhaft said:

I've heard of issues like this before, but I don't know what causes it specifically. It's best to go back to the player that first generated a 1MB file and have them redo their turn (that resulted in a 1MB PBEM file) and see if they generate an appropriate sized file.

I assume that even if you do nothing, the files should still be fairly large. Make sure that the directory that the PBEM files get written into is 'whitelisted/excepted' within whatever security/anti-virus software you use, just in case it has truncated/messed up the file as it was being written. Such interference, with this type of result, should be extremely rare, but it would be worth doing just in case.

We solved it by replaying the turn, for now it's not a big deal because at least on my end the orders are not complicated, but I assume that as the game progresses this will become more and more annoying. Are there any logs we can capture should we happen on this again? Or can you guys try and figure it out without our help? I have the scenario and turns if needed.

[BletchleyGeek]

Hi @Probus,

I get issues like this whenever the CM application loses focus while saving the file (as in when you alt-tab out of it). Sometimes this happens if you have Windows notifications popping up. Whitelisting the saved game folders for the Windows anti-malware service is a good idea, this issue also happened to me with the anti-malware service kicking in and slowing down disk operations to a crawl.

Edited April 13, 2021 by BletchleyGeek

[stikkypixie]

I assume the previous reply was meant for me. I can confirm that this has nothing to do with focus. In fact the last time it worked was when I accidently alt+tabbed. I believe this is a bug in the game, as I have vague recollection this happening in the original shock force as well. I extend my offer to provide files if this helps the debug process.

[BletchleyGeek]

Sorry @stikkypixiethat was indeed for you. I confused your avatar with that of someone else's.

[stikkypixie]

Update, since the last turn it's been impossible to get a correct turn written to disk. Whitelisting the folder does not help. @Schrullenhaft any other ideas? Can you do anything with the files? Or is there a place to report bugs, because right now I can't continue the PBEM game.

[Schrullenhaft]

So to recap; you were able to generate one or more turns successfully, by going back to the last valid PBEM file (50MB or so in size), but you have now experienced the same problem with the 1MB PBEM files, correct ?

Though it may not resolve anything, try again (with the ORIGINAL last good 50MB file your opponent generated). However this time both you and your opponent should issue NO NEW ORDERS when it is your turn to do so. See if the files maintain their size or again drop back down to the 1MB invalid files. If this has worked (the files are now back to the proper size), then issue the new orders, but keep track of EXACTLY what new orders you issue and to what units. Both you and your opponent will need to do this. If the problem returns then we can submit a bug report which would have the following:

1) Specs of each computer (OS/Windows version, CPU, memory, video card, video driver version).

2) The PBEM files, to include the first batch with the last good file and then the next file that was bad and then the second batch with its last good and first bad file.

3) A list (from each opponent) of the orders they issued right before the files went bad.

4) Both passwords for the PBEM game.

This is a bit tedious, but it may help find whatever may be the bug. The more we can document to consistently reproduce the bug, the greater likelihood that the problem can actually be addressed. I'll try to reproduce (with your files) what you've experienced with a clean (un-modded) install of the latest version of CMBS (what you have) and then report it to the bug-tracking system if I reproduce the problem. When it will get looked at I wouldn't know. Right now Charles is quite busy with the pending releases and other programming chores and I don't know his workflow of when he looks at the bug reports to resolve them.

Edited April 15, 2021 by Schrullenhaft
Incorrect bug reporting info.

[stikkypixie]

Just to be clear. What you want is a PBEM file from which you can try and reproduce the problem? I have such a file, and I can provide it to you now without having to involve my opponent. Note that the issue is not consistent, if we keep trying long enough eventually the game will give us the correct file.

If it's imperative that we follow your procedure exactly I will need to coordinate with my opponent. I understand that resources are limited, but it would be good to be able to have it looked at.

[Schrullenhaft]

We'll want the last file that was good and the next turn that was bad and any passwords needed to open either. We're hoping that the first file will generate a bad one consistently (or mostly so). The other details requested will be helpful, though unlikely to be a factor in the problem.

About how many attempts does it take to generate a good file ? Has the opponent confirmed that they are good (other than their size) ? Is this now happening consistently at this point in the game (i.e. - that you will generate bad files on every PBEM exchange) ?

I assume that this was a QB, is that correct ? If so, were you using one of the QB maps that came with the game or a custom one ?

Edited April 15, 2021 by Schrullenhaft

[stikkypixie]

The bad turns cannot be opened, as the game crashes during loading, so no need for the opponents' password. If you see the password prompt then it worked, otherwise the game crashes. I just tried it again replaying that turn, the first time I gave no orders and the turn was generated fine, the second time I called in a fire mission using the forward observer with the UAV and the output turn was not fine. So I think if you just give some orders and replay the turn 10 times you should see the corruption at least once.

Here is a link to the files (the scenario, a specs file, the original turn, the turn that was generated properly and the turn that was corrupt):https://www.dropbox.com/sh/3crnrz4yjoatjgm/AACoSEWZFeXGGYSNP-ReTgtla?dl=0

I will DM you the password.

I hope this helps.

[Schrullenhaft]

I'm unable to duplicate the problem with your Turn 40. Every turn I generate is fine (approx. 50MB file). I've tried no movement orders, a few movement orders, a few more movement orders and artillery/mortar fire adjustments and CAS requests. The only problem I had was when the game attempted to overwrite the same filename (I hadn't moved a file out of the 'outgoing' folder). The procedure I used was:

1) Load game, select 'Saved Games' and then the PBEM file.

2) Watched one turn of the playback (usually not moving the camera and not fast-forwarding through anything).

3) Issued orders of various types (I couldn't really call new fire missions since almost no one had any line of sight).

4) Saved file and exited game. Moved file from 'outgoing' to 'incoming' game folder.

5) Reloaded game and selected new PBEM file and 'cancelled' out once I reached your opponents password login. Exiting the game after this.

I had no problems with any saves (over 20). On average it took around 50 seconds for a file save. This was with a clean/un-modded install of CMBS 2.14 with the BattlePack. I'm running Windows 10 64-bit version 20H2. The system has an Intel i7-7700k, 32GB RAM and a GeForce RTX 2070 Super running the 456.71 driver (Sept 2020). The only things loaded were the Windows File Manager, a Firefox browser (with few extensions), Avast free edition (no exceptions for CM, to my recollection), Steam client, Malwarebytes (free edition), a Samsung SSD utility, Dropbox, MS One Drive (don't really use it - and files are NOT going to it), and a Garmin client (GPS).

Since the UAV was 'busy', I was unable to do anything with it. I wonder if that could have anything to do with it (if the problem is within CM). Beyond CM itself I can only think of looking for some software that could somehow be interfering with drive writes (typically this would be security/anti-virus software) or somehow inordinately occupying the CPU while the game is trying to write the file.

Edited April 16, 2021 by Schrullenhaft
Missing a few details.

[Vacillator]

13 minutes ago, Schrullenhaft said:

no problems

Sorry but I had to give you a like for 'Doofus Snafu-us'.  Plus you deserve more likes.

[stikkypixie]

3 hours ago, Schrullenhaft said:

I'm unable to duplicate the problem with your Turn 40. Every turn I generate is fine (approx. 50MB file). I've tried no movement orders, a few movement orders, a few more movement orders and artillery/mortar fire adjustments and CAS requests. The only problem I had was when the game attempted to overwrite the same filename (I hadn't moved a file out of the 'outgoing' folder). The procedure I used was:

1) Load game, select 'Saved Games' and then the PBEM file.

2) Watched one turn of the playback (usually not moving the camera and not fast-forwarding through anything).

3) Issued orders of various types (I couldn't really call new fire missions since almost no one had any line of sight).

4) Saved file and exited game. Moved file from 'outgoing' to 'incoming' game folder.

5) Reloaded game and selected new PBEM file and 'cancelled' out once I reached your opponents password login. Exiting the game after this.

I had no problems with any saves (over 20). On average it took around 50 seconds for a file save. This was with a clean/un-modded install of CMBS 2.14 with the BattlePack. I'm running Windows 10 64-bit version 20H2. The system has an Intel i7-7700k, 32GB RAM and a GeForce RTX 2070 Super running the 456.71 driver (Sept 2020). The only things loaded were the Windows File Manager, a Firefox browser (with few extensions), Avast free edition (no exceptions for CM, to my recollection), Steam client, Malwarebytes (free edition), a Samsung SSD utility, Dropbox, MS One Drive (don't really use it - and files are NOT going to it), and a Garmin client (GPS).

Since the UAV was 'busy', I was unable to do anything with it. I wonder if that could have anything to do with it (if the problem is within CM). Beyond CM itself I can only think of looking for some software that could somehow be interfering with drive writes (typically this would be security/anti-virus software) or somehow inordinately occupying the CPU while the game is trying to write the file.

Too bad you weren't able to reproduce it. You did more less what I did, the only difference is I moved the camera quite a bit and I am running the steam version with no mods. I checked the system resources and the system is barely under load, even under heavy load I would expect the file to be ok even if it takes longer. Appreciate you taking the time to look at it.

[Schrullenhaft]

So using the Turn 40 that you provided me and performing the number of moves/orders I did, do you still frequently get invalid/corrupt files ? It's possible I've only been able to manipulate so much of the game that I'm not causing it to output a corrupt file.

[stikkypixie]

I tried again today to see might be causing the issue. I first tried to record my actions with fraps, but in six attempts I tried the file generated correctly.

I then realized I did not have firefox and spotify running. So what I did was start firefox, go to youtube and leave video on pause. I also started spotify and left it on pause and went in to the game. The first time I tried to generate the file, the file was truncated (attempt 7) and the second attempt the game just crashed. I know that both firefox and spotify have hardware rendering and I wonder whether that might interfere with the game. I will try to see if I can narrow it down.

[stikkypixie]

For now making sure that firefox and spotify are not running seems to be working. Hopefully that's the fix.

[Suzuya]

To add, myself and a friend are currently experiancing a very similar issue. While trying to run a CMBS quick battle on the Steam version of the game we are able to start a quick match, buy our forces as normal and proceed to the first turn of the game. However the first player to go (we have both attempted this separately) can input their orders as normal but when saving the game it will create a file around 1kb in size for the save file. Any attempts to open the file result in an immediate crash, both of us have attempted to create the mission and each player had the same issue on both our machines.

I have attempted by myself a number of times to recreate the game using an altered version of the map and game settings to no success, for whatever reason the first turn will always result in a corrupted file. 

We were attempting to play on a slightly modified version of this https://www.thefewgoodmen.com/tsd3/cm-black-sea/cm-black-sea-add-ons-maps/chernova-hirka-mastermap-v1-0/ however i did also make a game based upon an unmodified version of the map just to make sure we had not made any issues while tweaking the map which also resulted in the same error. I had no issues creating a standard PBEM game of one of the games pre made battles but this is the first QB game we had started on the Steam version of the client. We have played similar sized scenarios before in CMBFN and both our machines are rather powerful so i had not assumed this map would have been an issue based on that.

[Heirloom_Tomato]

2 hours ago, Suzuya said:

To add, myself and a friend are currently experiancing a very similar issue. While trying to run a CMBS quick battle on the Steam version of the game we are able to start a quick match, buy our forces as normal and proceed to the first turn of the game. However the first player to go (we have both attempted this separately) can input their orders as normal but when saving the game it will create a file around 1kb in size for the save file. Any attempts to open the file result in an immediate crash, both of us have attempted to create the mission and each player had the same issue on both our machines.

I have attempted by myself a number of times to recreate the game using an altered version of the map and game settings to no success, for whatever reason the first turn will always result in a corrupted file. 

We were attempting to play on a slightly modified version of this https://www.thefewgoodmen.com/tsd3/cm-black-sea/cm-black-sea-add-ons-maps/chernova-hirka-mastermap-v1-0/ however i did also make a game based upon an unmodified version of the map just to make sure we had not made any issues while tweaking the map which also resulted in the same error. I had no issues creating a standard PBEM game of one of the games pre made battles but this is the first QB game we had started on the Steam version of the client. We have played similar sized scenarios before in CMBFN and both our machines are rather powerful so i had not assumed this map would have been an issue based on that.

There is a note on the TSD3 page about some troops left in the map. When you made adjustments to the map, did you remove those units? 

[Suzuya]

Yes we removed the units in the OOB, we forgot the very first time we tried and i thought that may have been the issue as well but unfortunatly it didn't seem to fix the problem.

[stikkypixie]

Just wanted to add that it might help to lower the graphics quality when this issue comes up