A bit of bad TCP/IP news

[aka_tom_w]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

Fuerte:

No, the whole point of the original system was to have no comparisons between systems. Each would do its thing all on its own. That is where the speed came from. So if one machine turns up 1.00001 and the other comes up with 1.00002 there is no way for the first machine to know that it must have 0.00001 added in order to be the same as the other.

The problem is very basic here. Generally, computations do not need to be predictable and exact to so many places left of the decimal. But the real problem here is that we can not ensure that each machine comes up with the same exact numerical values.

Something like the SETI program is, if I am guessing correctly, a sort of distributed prorcessing system. One machine creates some values and sends them to the host. The host is NOT independently processing the same information on its own, and therefore takes the guest's data "as is" and uses that. This is sorta the way PBEM works. And it is now how TCP/IP works for CM.

Yup, things have been totally recoded to work more like PBEM. We are testing it internally for at least the next week. Around Friday or so I will give you guys a new update on when we expect to release it.

Thanks,

Steve<HR></BLOCKQUOTE>

Great

Thanks for the update.

Have FUN testing

-tom w

[Guest Rob/1]

Cant wait to try it!

------------------

Game master of Robs CMC

Join or get more info on Rob's CMC today Email me at RobsCMC@hotmail.com

Rob

--------------------------

For more info on Rob's CMC go to Robs CMC HQHome of Robs CMC

Members of my CMC should join my CMC chat bord. Robs CMC Bord

BHQS bord

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Fuerte:

It would work. If all calculations would be done with double precision floating point (precision 15 digits), and all comparisons with single precision (7 digits, rounded correctly), for example, then it would work.

<HR></BLOCKQUOTE>

This won't work-- a simple square root operation can propagate an incorrectly rounded digit near the least-sig-fig of the double back to near the least-sig-fig of the single (but where it won't be rounded away), and then it won't give the same result on different machines. All numbers have to be consistently rounded by every mathematical operation that's done on them, i.e. you have carry extra (guard) bits around and perform a consistent (from machine to machine) rounding operation after each and every mathematical operation. This is described in the Sun document referenced above (scroll down to the stuff about IEEE 754). I'm not at all up to date on how chips or compilers treat this, but it wouldn't be surprising if you had to use software libraries that give you a performance hit if you want to be IEEE compliant.

------------------

Slayer of the Original Cesspool Thread.

[Fuerte]

<BLOCKQUOTE>quote:</font><HR>Originally posted by chrisl:

This won't work-- a simple square root operation can propagate an incorrectly rounded digit near the least-sig-fig of the double back to near the least-sig-fig of the single (but where it won't be rounded away), and then it won't give the same result on different machines. All numbers have to be consistently rounded by every mathematical operation that's done on them, i.e. you have carry extra (guard) bits around and perform a consistent (from machine to machine) rounding operation after each and every mathematical operation. This is described in the Sun document referenced above (scroll down to the stuff about IEEE 754). I'm not at all up to date on how chips or compilers treat this, but it wouldn't be surprising if you had to use software libraries that give you a performance hit if you want to be IEEE compliant.<HR></BLOCKQUOTE>

It would work. You can't have so big errors in double precision calculations (15 digit percision) that they would propagate to the seventh digit (where it would be rounded). Or if you insist, then give an example.

And good compilers like Microsoft Visual C++ are IEEE 754 compatible (to my knowledge). From MSDN:

"/Op (Improve Float Consistency)

The Improve Float Consistency (/Op) option improves the consistency of floating-point tests for equality and inequality by disabling optimizations that could change the precision of floating-point calculations.

By default, the compiler uses the coprocessor?s 80-bit registers to hold the intermediate results of floating-point calculations. This increases program speed and decreases program size. However, because the calculation involves floating-point data types that are represented in memory by less than 80 bits, carrying the extra bits of precision (80 bits minus the number of bits in a smaller floating-point type) through a lengthy calculation can produce inconsistent results.

With /Op, the compiler loads data from memory prior to each floating-point operation and, if assignment occurs, writes the results back to memory upon completion. Loading the data prior to each operation guarantees that the data does not retain any significance greater than the capacity of its type.

A program compiled with /Op may be slower and larger than one compiled without /Op.

Note The /Op option disables inline generation of floating-point functions. The standard run-time library routines are used instead. For more information, see the /Oi option.

If you select the Disable Language Extensions (/Za) option in order to compile for ANSI compatibility, the use of /Op is implied. The use of /Op improves the consistency of floating-point tests for equality and inequality. The nature of the improved consistency provides for strict ANSI conformance and is the only situation under which /Op is selected by default."

"/Op Ensures the precision of floating point operations. This is useful when you are concerned about error propagation and intermediate results of floating point operations. The following topics contain information on understanding floating point operations: IEEE Floating-Point Representation and Microsoft Languages and Why Floating Point Numbers May Lose Precision."s

"Microsoft Visual C++ is consistent with the IEEE numeric standards. There are three internal varieties of real numbers. Real*4 and real*8 are used in Visual C++. Real*4 is declared using the word float. Real*8 is declared using the word double."

"For EPSILON, you may use the constants FLT_EPSILON defined for float as 1.192092896e-07F or DBL_EPSILON defined for double as 2.2204460492503131e-016. You need to include float.h for these constants. These constants are defined as the smallest positive number x, such that x+1.0 is not equal to 1.0. Because this is a very small number it is advisable that you employ user-defined tolerance for calculations involving very large numbers."

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Fuerte:

You can't have so big errors in double precision calculations (15 digit percision) that they would propagate to the seventh digit (where it would be rounded). Or if you insist, then give an example.

<HR></BLOCKQUOTE>

Sure you can- I gave an example: the square root operation. Suppose you have a double whose correct value should be zero, but instead it ended up as 1x10^-15. Then you take a square root of it (since it may in some cases not be zero, you act on it like it's always got a value) and multiply by ten. It's value is now 3.162x10^-7. Then you add it to something called X. That X is now off by 3 in the 7th decimal place (the precision reduction will only get rid of the .162 stuff), since you should have been adding zero to it. BTS already gave an example where something in the 5th decimal place could have an effect. Given a slightly more complicated operation sequence you could move the error further into the more significant digits. Considering the complexity of the armor penetration formulas, it seems entirely possible for such things to happen. That's why the IEEE spec requires the guard digits and rounding for each op. It also only specifies behavior for a limited range of functions (which does not include transcendental functions, which probably do appear in the armor penetration formulas).

This is a contrived example, but such things are certainly possible. IIRC there's some examples of this in some of the intro material in "Numerical Recipes" as well, to warn the potential user that machine precision matters. I'm not at all expert in the arcana of floating point, and most of the above comes from a partial reading of the Sun document referenced above and my feeble memory.

<BLOCKQUOTE>quote:</font><HR>A program compiled with /Op may be slower and larger than one compiled without /Op.<HR></BLOCKQUOTE>

They may have tried this and decided it wasn't worth the hit, relative to modifying the PBEM style of multiplayer play, or some problem outside the scope of the IEEE spec (such as transcendental functions) may have reared it's ugly head.

(and this discussion should probably head off to some math newsgroup...)

------------------

Slayer of the Original Cesspool Thread.

[This message has been edited by chrisl (edited 11-12-2000).]

[109 Gustav]

Dang, after reading the last bunch of threads, I can't wait to learn C++. Anyway...

BTS, will the bug that causes Macs to freeze when the backside cache is enabled be fixed in the TCP/IP patch? I've bumped the RAM allotment up as far as my 128 megs can take, and it doesn't help. I've also run CM with the backside cache disabled, but it's painfully slow then.

Autosave is one of the best auxillary features I've seen in a game, but it doesn't help when my zook team makes a 130m kill on a tiger, and the game freezes.

------------------

No one but the enemy will tell you what the enemy is going to do. No one but the enemy will ever teach you where you are weak. Only the enemy tells you where he is strong. And the rules of the game are what you can do to him and what you can stop him from doing to you. -Ender's Game

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by 109 Gustav:

BTS, will the bug that causes Macs to freeze when the backside cache is enabled be fixed in the TCP/IP patch? I've bumped the RAM allotment up as far as my 128 megs can take, and it doesn't help. I've also run CM with the backside cache disabled, but it's painfully slow then.

<HR></BLOCKQUOTE>

What are you running on? I'm using a PB G3(wallstreet) w/1MB L2 cache with MacOS8.5 and don't have any problems with it siezing up (very very infrequently when I put in too many armor waypoints it hangs during orders, but it hasn't been predictable). I have a ton of RAM, but I think I've never given CM more than 80 or so, and I can run reinforced battalion sized games with no trouble.

And as for C++: I recently went to the trouble to learn the basics. The Dave Mark book that comes on the documentation disk with Codewarrior is a pretty good introduction. I can't remember if it's oriented towards people who already know C or not. From what I can tell, C++ looks like a great way to write truly indecipherable code if you're not very careful.

------------------

Slayer of the Original Cesspool Thread.

[This message has been edited by chrisl (edited 11-13-2000).]

[Guest Big Time Software]

<BLOCKQUOTE>quote:</font><HR>Sure you can- I gave an example: the square root operation. <HR></BLOCKQUOTE>

Funny enough, last week when I forwarded the stuff about the double precision stuff to Charles he said, right away, "yeah, but some functions like square root will screw that up". So me thinks chrisl is correct on this one.

Gustav, I am unfamiliar with the problem you are having. Do you have some sort of funky upgrade card or something? We PROGRAMMED Combat Mission on the Mac. First a 604e, then a G3 upgraded 8500, and now a G4. I also have always been using CM on the Mac, exclusively, and have never had a problem with either my G4 or my iBook. So whatever problem you are having, I don't think it is an accross the board sort of thing. In fact, this is the first I have ever heard about a problem like this, not that it means much to say this

Steve

[Smeltz]

I know nothing of your alien language. I come in peace. I shall wait at base until completion of the TCP/IP project. Updates have been recieved and noted. Awaiting fruther transmission. Out.

------------------

"Free your mind!"

[coralsaw]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

We PROGRAMMED Combat Mission on the Mac. <HR></BLOCKQUOTE>

OhmiGod! Conspiracy, Apple is silently gathering momentum.

Now I understand why you didn't put this, ahem, factoid on the box (come to think of it, I didn't get a box...).

I can see it happening. The giant ad all over the place: "All good software is developed on the Mac."

They want our blood, kill, kill!

PC owner (envious and bloodthirsty )

------------------

My squads are regular, must be the fibre in the musli...

[Fuerte]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

Funny enough, last week when I forwarded the stuff about the double precision stuff to Charles he said, right away, "yeah, but some functions like square root will screw that up". So me thinks chrisl is correct on this one.<HR></BLOCKQUOTE>

Sure chrisl is technically correct, but in practice the double precision floating point would certainly work. Just check for zero (with some epsilon value) before every squre root operation... there can't be so many.

"It is preferable to either treat a floating point field as character data that is immune to rounding errors or incorporate a small offset value into comparisons (known as an epsilon value) to allow nearly equal values to be considered sufficiently equal."

[109 Gustav]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

Gustav, I am unfamiliar with the problem you are having. Do you have some sort of funky upgrade card or something? We PROGRAMMED Combat Mission on the Mac. First a 604e, then a G3 upgraded 8500, and now a G4. I also have always been using CM on the Mac, exclusively, and have never had a problem with either my G4 or my iBook. So whatever problem you are having, I don't think it is an accross the board sort of thing. In fact, this is the first I have ever heard about a problem like this, not that it means much to say this

Steve<HR></BLOCKQUOTE>

I have an iMac DV 400 (old), with 128mb of RAM, running OS9. The ram is the only upgrade.

I thought there was a known issue of a conflict with the backside cache, but I guess I might be wrong.

I gave CM 60,000mb of RAM, which is all I really could without running virtual memory. It still tends to freeze on large maps. I think it freezes most often when I move the camera around the map a lot on one turn, like when I'm plotting orders for a bunch of units in different places.

I've noticed that no matter how much I scroll the camera around, CM never freezes when I'm using the map editor.

Glad to hear that CM was programmed on a Mac. I was wondering about that.

------------------

No one but the enemy will tell you what the enemy is going to do. No one but the enemy will ever teach you where you are weak. Only the enemy tells you where he is strong. And the rules of the game are what you can do to him and what you can stop him from doing to you. -Ender's Game

[Major]

Thanks for the message.

[GriffinCheng+]

BTS, may I ask if Charles is using Code Warrior for the developments for both PC and Mac? Well, frankly I know little C/C++. Java is more of my cup of tea -- ah, coffee.

Griffin.

------------------

"+" is just the beginning. Expect to see "GriffinCheng76", "GriffinCheng(105)" or "GriffinChengA3E8" more should Forum problems occur again

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Fuerte:

Sure chrisl is technically correct, but in practice the double precision floating point would certainly work. Just check for zero (with some epsilon value) before every squre root operation... there can't be so many.

<HR></BLOCKQUOTE>

It's still not sufficient-- that's just one example of error propagating into the more significant digits, but not the only possible case. If you'd read the tech note published by Sun, and the appendix (also referenced above) but again here: http://www.validgh.com/goldberg/addendum.html , you'd see that even two different implementations that are IEEE compliant can give different results even when using exclusively functions that are within the scope of the IEEE spec, which is recognized by its authors to be incomplete, but much better than nothing. CM almost certainly also uses functions that are outside the IEEE spec (the exponential for example).

Instead of rewriting a whole ton of code, which would take a good deal of time (and in the meantime be rendered moot by everyone's increase in available bandwidth), and which would probably still result in some inconsistencies in computation (if it were that easy the IEEE would likely have specified more), BTS is taking the sure road to success (at slight bandwidth cost) and using a system that is guaranteed to produce self-consistent results.

------------------

Slayer of the Original Cesspool Thread.

[Guest Big Time Software]

chrisl, this is my understanding from Charles as well. First he would have to KNOW, for a fact, every function that could yield a messed up rounded value. Then he would have to know exactly how to rig each one so that it would do things consistantly. But it is unclear if that can even be done. This would be a whole lot of effort and STILL there would be grave questions about stability. No, far better to go with what we know will work 100% of the time and only took about a day to code

Griffin, yes Charles is using CodeWarrior for both platforms. He has few complaints about it, although like any compiler it sometimes likes to do things the wrong way

Gustav, first I heard about a potential problem with the backside cache. Sounds more like you are overwhelming the video card.

Steve

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

chrisl, this is my understanding from Charles as well. First he would have to KNOW, for a fact, every function that could yield a messed up rounded value. Then he would have to know exactly how to rig each one so that it would do things consistantly. <HR></BLOCKQUOTE>

And then he'd probably have to write it up and send it to an applied math journal, which would further delay CM2...

From the little that I've read it sounds like IEEE tried to specify the things that could be provably controlled, or at least contained within a provable error limit, and wouldn't touch anything else with a pole of arbitrary length.

------------------

Slayer of the Original Cesspool Thread.

[Seahawk-vfa201]

Sorry for jumping in on a a topic that has been covered extensively.

I just got the news that the newly approved C99 standard defines a new header file called < inttypes.h >, which defines sets of typedefs of integer types.

One of these sets defines integer types with certain exact widths:

int8_t int16_t int32_t int64_t

And their unsigned counterparts are

uint8_t uint16_t uint32_t uint64_t

The names are rather intuitive. For example, int8_t is a signed

integral type that occupies exactly 8 bits. Likewise, uint8_t is an

unsigned integral type that occupies exactly 8 bits. Using these typedef

names you'll get integers whose widths are identical on every platform.

Of course, if the problem is on how rounding is performed on different platforms than it might still exist.

I thought anyway that the news could be of some interest in this topic.

Cheers

[tss]

Seahawk-vfa201 wrote:

I just got the news that the newly approved C99 standard defines a new header file called < inttypes.h >, which defines sets of typedefs of integer types.

Hmm. It will be interesting to see what happens to the header name when it gets passed through the post formatter.

Anyway, integer arithmetic is exact and by its very nature it doesn't have any rounding problems. The number of bits available in a word is not really important, if the lower bound (= 32 bit) of the supported systems has enough.

But the problem was with floating point numbers, which have much much much different behavior than integer have. My copy of C99 draft is in some unknown location so I can't check whether it would give any help in this case but I seriously doubt that. First, it is probable that the floating point numbers will be IEEE floats with all the problems that have been mentioned in this thread. Second, there are _no_ C99 compilers in existence right now. So, even if it would help, it will take at least a year before first usable C99 compilers are out.

- Tommi

[109 Gustav]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Big Time Software:

Gustav, first I heard about a potential problem with the backside cache. Sounds more like you are overwhelming the video card.

Steve<HR></BLOCKQUOTE>

Hmm, good point. I'll try running at a lower res. That'll probably do the trick. Thank you for the help. It still amazes me that a game company can interact so much with the gamers, instead of just handing down updates and decsions from above.

------------------

No one but the enemy will tell you what the enemy is going to do. No one but the enemy will ever teach you where you are weak. Only the enemy tells you where he is strong. And the rules of the game are what you can do to him and what you can stop him from doing to you. -Ender's Game

[Boris Balaban]

Sorry I have not read every posting but is BTS any closer to getting TCP/IP up and running?

------------------

Boris Balaban

WORK bbalaban@rim.net

HOME b.balaban@home.com

[aka_tom_w]

<BLOCKQUOTE>quote:</font><HR>Originally posted by Boris Balaban:

Sorry I have not read every posting but is BTS any closer to getting TCP/IP up and running?

<HR></BLOCKQUOTE>

Simple....

Steve Says:

"Yup, things have been totally recoded to work more like PBEM. We are testing it internally for at least the next week. Around Friday or so I will give you guys a new update on when we expect to release it.

Thanks,

Steve"

and that was the last I have heard.

-tom w

[degreesK]

With the help of a friend with a Mac I've managed to get the following chaotic function to give the same results on both Mac and PC for doubles. Wasn't able to get the same results when using floats. PC gives the same results on AMD/Intel and Win98/2000.

#include <stdio.h>

#include <float.h>

int main()

{

float f;

double d;

int i;

f = 0.5;

d = 0.5;

for (i = 1; i <= 200; i++)

{

f = 3.8f * f / (1.0f / (1.0f - f));

d = 3.8 * d / (1.0 / (1.0 - d));

if (i % 10 == 0)

printf ("%10d %20.5f %20.5f\n", i, f, d);

}

getchar();

}

after the 200th iteration the result is .91816 on both PC and Mac for the double ( d ).

I'll try getting square roots working next.

[degreesK]

Oh yeah, the function above usually deviates around the ~60th or so iteration if there's an accuracy problem. We just double checked 20 000 iterations and both the PC and the Mac gave the same results. Using Visual C++ 6 on the PC and Codewarrior Pro 6 on the Mac.

[chrisl]

<BLOCKQUOTE>quote:</font><HR>Originally posted by degreesK:

With the help of a friend with a Mac I've managed to get the following chaotic function to give the same results on both Mac and PC for doubles.

I'll try getting square roots working next.<HR></BLOCKQUOTE>

That's not too surprising, since it's all arithmetic operations and probably should be standardized. Even the square root might not generate differences (though once they're there it can propogate them). The problem will be with things like trig functions and exponentials. And experiments like that are interesting, and valuable if you're only running on a limited set of machines with a limited set of functions, but since CM is a ton of code, you really would rather have a formal proof (in the mathematical sense) that the code will generate the same numbers on all systems. That would be a major pain, and it sounds like the TCP/IP stuff is well into test already with a system that will work unquestionably (as far as producing the same results for both players).

------------------

Slayer of the Original Cesspool Thread.