PBEM File Size

[MagicBean]

I have started my first PBEM game. I am playing Death of Titans with a friend. The file sizes are large, after 3 turns they are up to 1.6 meg. Zipping helps a bit.

Is this normal?

Is there anyway of PBEM with smaller files?

T.

[Firefly]

1.6 mb is pretty large, you are probably playing a large scenario or QB with a lot of armour - in my experience moving armour is what bumps up the size. The size of the files will probably shrink as the game progresses and AFVs start dying. The only answer is to play smaller scenarios with less vehicles if it's a problem for you, I've seen small infantry only encounters with movie turn file as low as 40k.

[Ozzy]

Yes, that is a quite normal size.

I have experienced PBEM file sizes from 150 KB up to 2.5 MB.

I use RAR instead of ZIP, has some slightly better results.

The file size largely depends on unit actions such as movement and fighting/firing/targeting, etc. Hence, during the first turns, when you are still at full strength and almost all units are moving to contact, the file sizes are the largest.

Usually, with decreasing unit strength and movement the file size gets smaller.

The only way to keep the sizes small is a) play small or tiny battles or do not move your men ....

[Viceroy]

Yes, I find it's the amount of armor that really increases the PBEM file size. So, a scenario can be described as "small" in terms of map size, but if it has heaps of Tanks, HTs etc the file will be pretty big.

[ March 19, 2002, 01:24 PM: Message edited by: Viceroy ]

[Redwolf]

Just a quick note, compressing the PBEM files with no matter how good compression program makes them bigger for transfer purposes.

[Dirtweasle]

Redwolf - can you explain that? Compresing the files makes them biggeer for transfer purposes?

[JonS]

Turning off smoke and trees may help make the files a little smaller.

[Pak40]

Originally posted by Dirtweasle:

Redwolf - can you explain that? Compresing the files makes them biggeer for transfer purposes?

I think he means that the original PBEM is an ASCII file but The zipped PBEM file is binary; but in order to email the file it has to be converted to ASCII (seemlessly done by your email program). The ASCII conversion will be larger in size than the binary file. So when all is said and done, there really isn't much of a gain in zipping your PBEM file, you may actually be increasing the file size.

[Redwolf]

Here's a preview of an article:

Why not to zip/compress PBEM moves

To understand the issue, you have to know that practically all

email transportation on the internet is done in ASCII form, not binary

form. The protocol SMTP, which is used for all normal email

transportation on the internet requires it, and the protocols pop3 and

imap which often deliver the mails to the enduser aren't any better.

The PBEM file as generated by Combat Mission is an ASCII file. The

actual data that CMBO uses is binary, but for your convenience it

wraps it into ASCII. The data before being converted to ASCII is

already compressed (in fact it is encrypted and then compressed), so

what your really have is binary CMBO data which is first compressed

and then made into ASCII.

If you zip such a CMBO move, it gets smaller:

<table>

<tr><th>size<th>filename

<tr><td> 990023<td> nordic-cbg-rwolf056.txt*

<tr><td> 749052<td> nordic-cbg-rwolf056.zip

</table>

That's nice, it saves 25% - while resting on your harddrive that is.

However, if you actually mail the zipfile, your mailer is

converting it to ASCII again. A zipfile is not ASCII, it is binary,

that means any value a byte can have mixed freely. The usual email

protocols don't allow real binary to be transported since they are too

braindead to tell the extend of the file then, among other protocol

weaknesses. So your mailer ensures that everything is ASCII.

Everything - if it isn't ASCII to start from, it converts it to ASCII,

without mentioning it to you and the reader's mailer converts it back,

again it goes without saying.

That means the original CMBO file is transported as-is, but the

zipfile is converted first.

So what does the conversion to ASCII look like? There are two

methods in common use - uuencode and base64. Base64 is a little more

efficient so it is in use in most mailers today. Still, although

base64 does the best job it can, it operational requirements are

pretty limited. It has to wrap everything in the few "real" ASCII

chars (not counting umlauts and stuff) and it inserts newlines to make

it look like a real formatted text.

So what is the size for uuencode (*.uue) and base64 (*.base64)?

<table>

<tr><td> 990023 <td> nordic-cbg-rwolf056.txt

<tr><td> 749052 <td> nordic-cbg-rwolf056.zip

<tr><td> 1026480 <td> nordic-cbg-rwolf056.zip.base64

<tr><td> 1032068 <td> nordic-cbg-rwolf056.zip.uue

</table>

As you can see, the ASCII conversion did not only eat up the zip

advantage, it made things worse.

And that is natural. The original CMBO file is already a

compressed file before it was turned into text, and compressing a

compressed (or encrypted) file again will make it bigger (unless the

first compressor was exceptionally braindead). The zipping of the

original file could do some compression because it squeezed out the

ASCII wrapper of the CMBO file, but when you later add a new ASCII

wrapper for email transportation, you get the overhead that is natural

when compressing already compressed files. You gained nothing from

first compressing an ASCII wrapper or to re-wrap later, and on top of

gaining nothing there you have two compression overheads and

uneliminated traces of the first ASCII wrapper which you don't even

use anymore.

<h3>Attachment damage</h3>

People frequently zip PBEM file to overcome email size limitations. As

we have seen, that is nonsense.

But often people zip PBEM files because they find the text

attachment is damaged while being transported, but zipfile attachments

are being transported OK.

That is a result of bad providers trying to convert text

attachments (the CMBO file looks like a text attachment) "for your

convenience" and getting it wrong. New linebreaks, character

conversions, splitting etc. is all deadly for CMBO files. Frequently

these providers do not touch declared zipfile attachments, but

declared text attachments. You original CMBO file is being transported

with a declaration of being a textfile.

This is all fine, but the only answer is to dump the provider and

get a decent one. This is in no way acceptable. Your emails have to be

transported as-is, with no modification except the required envelope

changes on SMTP transportation. Get a new provider instead of

switching to zipfiles which causes more work for everyone and makes

the actual transmitted emails bigger.

<h3>So you really want it small?</h3>

If you really want these 25% reduction to apply, because you pay big

bugs my megabyte or your throughput is very low, you can do that by

not using email. It requires direct connections (both sides dialed

up), unless you have a third-party server to place stuff at.

Use a protocol like ftp or HTTP. Install a small webserver, place

the move there and let the other guy get it with his/her browser or

automated tool. ftp allows two-way getting and placing, but transmits

password in clear - do not use important passwords and shut down the

service after use. If you want something decent, get an implementation

of the ssh client and server. Then use their scp program or their

secure ftp replacement.

I have to remind you that a Windows machine with any recent version

of Internet Exploder allows anyone to do anything on your machine

(including password snarfing) anyway

And it don't want people to say that I am overly negative, but what

can you do with these providers and Microsoft and awful Hollywood

movies...

[Firefly]

Originally posted by JonS:

Turning off smoke and trees may help make the files a little smaller.

Not really because you are only turning it off for yourself. The graphics (or the code for them) are still there.

[MagicBean]

Could we play a hot seat game and mail each other our saved games?

An idea but I supect it wont work?

T.