The (Possibly) Unbalanced Tournament Scoring System

[Nabla]

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

Even simpler, what if the "curve" was linear all the way from -100 to 100 at a low slope, say 4:1?

<hr></blockquote>

A lot of stuff here, but I have some social responsibilities to attend to right now so I'll just comment on this one shortly. In a previous post you've also made a similar suggestion that the curve should start out more flat. I think - I'll try to prove this another day - that because finally the scoring system is used only to order the players from worst to best, we can multiply the slope with any positive value and we still get the same ordering. The differences between the final scores will be different, but the ordering will be the same.

Therefore I think that a curve with a 4:1 slope all the way produces the same order as a curve with a 1:1 slope. This is also the reason why we can agree to always start with a 1:1 slope in those curves in which the slope decreases.

Ok, social responsibilities can wait for one minute, here's an informal proof. Let i and j be game indices, and a_i and b_j denote the differences from median for players A and B in games i and j (here _ denotes subscript). Now let f(a_i) denote final score for curve f, and sum_i f(a_i) be the final score of player A (sum over scores of all individual games). If sum_i f(a_i) > sum_j f(b_j), player A is better than B using curve f. If we multiply f by some positive number, say X, the clearly still sum_i X*f(a_i) > sum_j X*f(b_j). So the ordering is the same.

[Treeburst155]

I understand what you mean now that I think about it. Any linear "curve" is just like any other linear "curve" in regards to the final outcome of the tourney. Players' point differences would be much closer with a low slope, but points would be more difficult to get too. IOW, overtaking somebody ahead of you would be no easier with a shallow slope than with a steep slope. It's this part I failed to realize when I brought up the idea. Let's eliminate anything that is linear in nature from the discussion.

For the same reasons I understand why 1:1 is fine around the median. It's what we do before and after the median in relation to that 1:1 that matters. In fact, I believe the actual ratio around the median does not matter much. What really matters is how the ratio at the median compares with the rest of the curve on both sides.

When you get a chance, give my your thoughts on my other suggestion, the one with linearity at 2:1 from -100 to +10 with the curve flattening out as victory levels increase. (Victory being defined in terms of relation to the median).

EDIT: You know, Nabla, you are much more qualified to determine a curve that accurately measures a player's skill than I am. That's why we call it "The Nabla System".

Let's restate our goals for the system:

1) Overwhelming victories should not be too heavily rewarded for the following reason:

A) Big victories could very well be the result of extremely poor play on the part of the opponent, such as surrendering or merely playing for "fun" once the battle turned against him.

2) Players should reap noticeable benefits from a successful withdrawal. This encourages more realistic play IMO. In CM players tend to surrender just to be done with a "failed" game.

3) Related to #1 above, the curve should yield a competitive tight race. One big loss or victory should not take a player out of the running or win the tournament for him.

4) The unbalanced nature of virtually all scenarios should be accounted for. (The median does this)

5) Players who are winning a game Big Time should have sufficient incentive to get all the points they can.

I think if the curve was fairly steep on the negative side of the median and shallower on the positive side, but never becoming virtually flat, we would achieve what we want. When deciding on this curve it should be kept in mind that only rarely will any portion of the curve >|40| be utilized. The vast majority of scores from both sides of a scenario will be <|30| from the median for that side.

Treeburst155 out.

[ 12-19-2001: Message edited by: Treeburst155 ]

[ 12-19-2001: Message edited by: Treeburst155 ]</p>

[Nabla]

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

I understand what you mean now that I think about it. Any linear "curve" is just like any other linear "curve" in regards to the final outcome of the tourney. <hr></blockquote>

Exactly.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

For the same reasons I understand why 1:1 is fine around the median. It's what we do before and after the median in relation to that 1:1 that matters. In fact, I believe the actual ratio around the median does not matter much. What really matters is how the ratio at the median compares with the rest of the curve on both sides.

<hr></blockquote>

Exactly 2 A 1:1 start ratio is just easy to compare with a neutral 1:1 linear curve.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

When you get a chance, give my your thoughts on my other suggestion, the one with linearity at 2:1 from -100 to +10 with the curve flattening out as victory levels increase. (Victory being defined in terms of relation to the median).

<hr></blockquote>

I will do that.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

EDIT: You know, Nabla, you are much more qualified to determine a curve that accurately measures a player's skill than I am.

<hr></blockquote>

Definitely not! I'm your analyst. (Reading this message through once more I had to add that I am still happy to see that our opinions on the priorities agree practically completely. ) I can tell you what can be done, and pinpoint pitfalls, locations where problems might arise. But you're the customer. You have much better knowledge about tournaments, knowledge of what we finally want to achive with this system. And this cooperation has worked very well It's been very easy for both sides to understand what the other person has been saying. This is quite an accomplishment since we can only interact through this board.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

Let's restate our goals for the system:

1) Overwhelming victories should not be too heavily rewarded for the following reason:

A) Big victories could very well be the result of extremely poor play on the part of the opponent, such as surrendering or merely playing for "fun" once the battle turned against him.

2) Players should reap noticeable benefits from a successful withdrawal. This encourages more realistic play IMO. In CM players tend to surrender just to be done with a "failed" game.

3) Related to #1 above, the curve should yield a competitive tight race. One big loss or victory should not take a player out of the running or win the tournament for him.

4) The unbalanced nature of virtually all scenarios should be accounted for. (The median does this)

5) Players who are winning a game Big Time should have sufficient incentive to get all the points they can.

I think if the curve was fairly steep on the negative side of the median and shallower on the positive side, but never becoming virtually flat, we would achieve what we want. When deciding on this curve it should be kept in mind that only rarely will any portion of the curve >|40| be utilized. The vast majority of scores from both sides of a scenario will be <|30| from the median for that side.

<hr></blockquote>

See, this is what I was talking about. You have stated here exactly what I wanted to know. This is how it works.

Now, with these clear priorities, I will retreat to my study room for a while to think about the situation.

[ 12-20-2001: Message edited by: Nabla ]</p>

[Redwolf]

Did I understand your correctly that in an overwhelming victory

- the looser is punished quite heavily, so that he is motivated to try a retreat from the map

- but the winner doesn't gain very much, just barely enough to keep him motivated to maybe try a few points more

If so, that sounds exactly like the right thing!

[Nabla]

<blockquote>quote:</font><hr>Originally posted by redwolf hates artillery:

Did I understand your correctly that in an overwhelming victory

- the looser is punished quite heavily, so that he is motivated to try a retreat from the map

- but the winner doesn't gain very much, just barely enough to keep him motivated to maybe try a few points more

If so, that sounds exactly like the right thing!<hr></blockquote>

Yes, you have the basic idea. In the current plan the punishment on the looser is not heavy, but it is significant. In addition, part of the motivation for the winner will be to try to sink the loser a bit lower. Well, that's a nasty way of putting it, a more political way is to say that the winner tries to maximize his chances of winning the whole tournament by reducing the losers chance of winning.

I'm glad to hear that you agree. Even if you didn't agree, I'd still be glad to hear your opinion. This system will probably/hopefully affect a number of tournaments, so input is very welcome at this point.

[Treeburst155]

OK, Nabla, go to your study and put that wonderful brain of yours into high gear. I'll let you out in a few days.

Treeburst155 out.

[Nabla]

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

When you get a chance, give my your thoughts on my other suggestion, the one with linearity at 2:1 from -100 to +10 with the curve flattening out as victory levels increase. (Victory being defined in terms of relation to the median).

<hr></blockquote>

I think this would violate rule number 3:

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

3) Related to #1 above, the curve should yield a competitive tight race. One big loss or victory should not take a player out of the running or win the tournament for him.

<hr></blockquote>

If the curve was at its steepest all the way down to -100, a player who lost Big Time in one game would be practically out of the tournament. In order to compensate for this big loss he would have to win a large number of games. A few large victories would not do it since the curve is much flatter on the positive side.

Do you agree?

[Nabla]

I think that the asymmetric curve fulfills all of our current goals. Let's go through these one at a time.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

1) Overwhelming victories should not be too heavily rewarded for the following reason:

A) Big victories could very well be the result of extremely poor play on the part of the opponent, such as surrendering or merely playing for "fun" once the battle turned against him.

<hr></blockquote>

The very flat (exponential) curve on the positive side takes care of this extremely efficiently.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

2) Players should reap noticeable benefits from a successful withdrawal. This encourages more realistic play IMO. In CM players tend to surrender just to be done with a "failed" game.

<hr></blockquote>

The not-extremely-flat (asinh) curve on the negative side rewards successful withdrawal. (However, whether the benefit is "noticeable" should be analyzed more carefully using some examples.)

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

3) Related to #1 above, the curve should yield a competitive tight race. One big loss or victory should not take a player out of the running or win the tournament for him.

<hr></blockquote>

Since the curves on both sides are relatively flat the system takes very good care of this once we can define a correct value for a (the degree of flatness / the point where the curve turns flat).

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

4) The unbalanced nature of virtually all scenarios should be accounted for. (The median does this)

<hr></blockquote>

Agreed.

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

5) Players who are winning a game Big Time should have sufficient incentive to get all the points they can.

<hr></blockquote>

This is difficult, since it so easily conflicts with goal 1. As noted above, in addition to the minor direct incentive of getting a few more points, the asymmetric system creates a second incentive by lowering the score of the loser significantly if the winner gains more points. I think it's a very nice feature. Whether or not it will motivate the winners in real tournaments is a whole different question.

So at this point I would say that we should go with the asymmetric system. If you agree, then I suggest that we will do the following:

1. A home exercise for me. The use of the asymmetric curve requires doing some math and some programming work. I have to compute the formula that relates the two parameters, and implement this into the program. The person running the tournament will only give parameter a for the flat curve on the positive side, and the program will compute the corresponding parameter for the negative side. The point of this computation is to a) ensure that the two curves have almost identical shapes near zero and define a reasonable point from which the two curves differ.

2. After the program has been written, we should generate a few results for imaginary tournaments, try to define a good value for the parameter a, and see how the whole thing works.

What do you say, is this the plan?

[ 12-21-2001: Message edited by: Nabla ]</p>

[Treeburst155]

I think this is an excellent idea! I can help substantially with part two of the plan by plugging my Wild Bill results into the next formula. I assume I will be able to change the variable for the "a" parameter as in the earlier formula?

As far as sufficient incentives to make big victories bigger, maybe the best we can do there is punish the loser. The big winner might not gain many more points, but he at least knows he's punishing the loser as his degree of victory increases still further.

OK then, you go back to your study and come up with the new asymmetrical formula. When it's ready I'll start running hypothetical tourneys (and real ones) through the formula and play with the "a" parameter. Then we will meet here and discuss. It's gonna be good!!

Treeburst155 out.

[Nabla]

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

OK then, you go back to your study and come up with the new asymmetrical formula. When it's ready I'll start running hypothetical tourneys (and real ones) through the formula and play with the "a" parameter. Then we will meet here and discuss. It's gonna be good!!

<hr></blockquote>

This is a good plan. I'll get back to you when the first part is done. And a merry Christmas preparation time for everyone. :cool:

[Nabla]

Time to dig up this thread from dust again.

So we've got one last thing to take care of: the actual mathematical definition of the asymmetric curve. Already during Christmas I started to think about an optimal way of computing the curve, but it became way too difficult. So I came up with a very straightforward way of implementing our idea.

This is my suggestion. When the tournament manager runs the scoring program he gives it three options. The two first ones are as before: type of nonlinearity used (either exp or asinh) and the single parameter of this nonlineary which determines the flatness of the curve. But the third parameter is new: it is the minimum slope of the curve on the negative side. Let me give you an example. Let's say we use the exponential nonlinearity, and a=.05, and the minimum slope is .5. Then the curve looks like this (for visual purposes only the part covering -60 to 60 difference from median is shown).

The curve on the negative side follows the shape of the nonlinearity on the positive side until its slope reaches this minimum value. From that point on the curve has a constant slope.

I don't know whether this is the best solution to the problem, but it's easy to implement and has only one extra parameter to play with. What we should remember all the time is that we will probably never see a deviation of -100 from the median. Let's say that in a tournament there is one game with median (80,20). One guy plays it bad, frustrates, surrenders and the game ends (20,80). The difference from median is 60. With the curve shown above the winner gets a final score of 19, while the loser gets -33. So with two big wins he's back in the game.

Then again, if the minimum slope is .5, like above, then the asymmetry of the curve does not affect scoring for small deviations from the median. For example, for a difference of 20 points from the median the positive score is 12.6, while the negative score is -13.1.

I know it's very difficult to state an opinion if you can't play with this system, so I'll start implementing it right away (it shouldn't take long, will be available latest tomorrow). I'll post a message here once the new version is ready for downloading.

[ 01-26-2002: Message edited by: Nabla ]</p>

[gredeker]

Nabla,

I don't even pretend to understand the formula, but could you run it on one of the scenarios that has been completed in the ROW tourney? That way we can see how it would work on some real life data.

Thanks.

[JPS]

I do not really wish to rush anyone, _but_ it would be extremely non-desirable situation if last minute tweaking of the scoring system parameters decides the finalists in RoW tournament.

As most of the battle scores are already in, it is feasible to make reasonable educated guesses of what parameters would benefit the standing of certain players.

I would suggest the following: Nabla proposes the most reasonable set of parameters (based on his current understanding of the scoring system) to Treeburst, and Treeburst makes his final parameter decision with no further input from other participants. IMO, this should be done with no further delay.

JPS

[Nabla]

<blockquote>quote:</font><hr>Originally posted by JPS:

I do not really wish to rush anyone, _but_ it would be extremely non-desirable situation if last minute tweaking of the scoring system parameters decides the finalists in RoW tournament.

As most of the battle scores are already in, it is feasible to make reasonable educated guesses of what parameters would benefit the standing of certain players.

I would suggest the following: Nabla proposes the most reasonable set of parameters (based on his current understanding of the scoring system) to Treeburst, and Treeburst makes his final parameter decision with no further input from other participants. IMO, this should be done with no further delay.

<hr></blockquote>

I agree. I was going to post a message with a similar idea today. At least we can't let any of the players in WBWRW participate in this discussion.

I will post no further information about this subject to this board until the final parameters for WBWRW have been chosen. I will discuss their selection with Treeburst155 directly via email.

[JPS]

And just to make my situation clear: I am one of the participants, and as I have finished all my battles, I have seen the current scores. Thus I won't provide any further opinions with regard to either scoring system or reasonable parameters.

[Treeburst155]

It is a bit dicey with us using the actual tourney scores to decide on the formula parameters. It raises the question of me possibly manipulating the parameters to achieve a certain desired outcome in the Wild Bill tourney. I don't think I want this question popping up in the participants' heads. For this reason I will use the original Nabla curve I spelled out in detail on the WBW Tourney thread early on. The one fairly recent change I will implement is the splitting of points for contested VLs. Players will be more at ease with the results if the scoring method used was in place before the results were known.

Let's continue work on the assymetrical curve using the WBW results (nothing beats actual results). We can use the new curve for the Nordic Championship, Nordic Wannabee, and future tournaments. This also gives us more time.

Treeburst155 out.

[Nabla]

<blockquote>quote:</font><hr>Originally posted by Treeburst155:

It is a bit dicey with us using the actual tourney scores to decide on the formula parameters. It raises the question of me possibly manipulating the parameters to achieve a certain desired outcome in the Wild Bill tourney. I don't think I want this question popping up in the participants' heads. For this reason I will use the original Nabla curve I spelled out in detail on the WBW Tourney thread early on. The one fairly recent change I will implement is the splitting of points for contested VLs. Players will be more at ease with the results if the scoring method used was in place before the results were known.

Let's continue work on the assymetrical curve using the WBW results (nothing beats actual results). We can use the new curve for the Nordic Championship, Nordic Wannabee, and future tournaments. This also gives us more time.

Treeburst155 out.<hr></blockquote>

Ok, loud and clear. This is the way it will be done. I will leave the question regarding Nordic Championship and Nordic Wannabee scoring parameters to your hands for now. We should probably decide those parameters before the first rounds of these tournaments are finished.

Just in case anyone else is interested, the programs can be downloaded from here.