The "Analysis of Nabla scoring system" thread

[dieseltaylor]

I think I may have cracked a fundamental concept of where the Nabla is malfunctioning.

The Nabla works fine where all people play the same sides continually - 5 games as the Axis the players will all of have received the benefits/losses from Nabla system for the same scenarios.

The problem arises when you mix Allied and Axis players in sections who effectively are playing on a different reward risk scale to each other. All of a sudden the scenario and side you draw can be crucial - particularly where the swings in scenarios can be as much as 5.90 or as little as 3.72 [winner 2.95 loser-2.95 etc].

If you all in a section play German and all in another section play Allied then the winner of each section has, apart from quality of opponent , played exactly the same games as each other player in his section and is therefore truly the winner.

Being a section winner would be an achievement in itself. The best x players from each side can then fight it out.

It seems so basic a concept- I have I missed something?

[JonS]

Had this thought last night:

1) Sum all raw scores from each side.

2) Divide each indiv. players raw score by amount from step 1).

3) Multiply each figure from step 2 by some 'scaling factor'. Could be 1, could be 10, could be 31 (1 point for each player), it doesn't really matter - though it should be the same for all scenarios.

The result of step 3) is your NABLA score for that side, for that scenario. The scaling factor is the total pot you are playing for, and the better you do in comparison to the other players of that scen/side combo, the more of the pot you get for yourself.

So, using the ROW V results for Malame and a 'pot' of 100 we get:

JonL (Best Axis, raw score 96) 3.9

BigDog944 (Worst Axis, raw score 10) 0.4

Walpurgis (Best Allied, raw score 90) 7.9

Jeff Wilders (Worst Allied, raw score 4) 0.4

In both cases (Axis and Allied) the total of points awarded to each side is 100, but for the Allies a bigger chunk is concentrated amongst a few players who did very very well (eight Allied players score higher than the max Axis score of 3.9), while the 100 Axis points are more evenly distributed amongst all the Axis players.

I have a spreadsheet here showing the working if anyone's interested ...

[ June 12, 2005, 12:14 PM: Message edited by: JonS ]

[JonS]

BTW, in ROW V at least 2 players played Axis in all 5 games. I had a mix of sides, but was attacker in all 5 games.

[dieseltaylor]

I understand, I think, what you are saying. I do not disagree other than to say you still show the Axis player as being restricted in total possible points earned. So you could have the biggest Axis winner in a section with the two highest Allied winners and he would be 4 points in arrears!

If they are in the same section this is not equitable - the nationalities must be by section for Nabla to really work properly.

[Malakovski]

[[edit: man you people post fast! this was a reply to Diesel's "I've cracked it" post a few back]]

That would, indeed, address the issue completely, although it would make administering things a little harder.

But really, I think it would be best to have this discussion after the tourney concludes. ROW V is going to be scored with the old nabla system, so there's no need to discuss revision until afterwards.

Apparently some people are sore about the results, and this discussion just seems to be delaying the point at which everyone accepts things and moves on.

[dieseltaylor]

Well I thought about it - then I thought what happens if I die in the meantime!!!

[JonS]

Originally posted by dieseltaylor:

I understand, I think, what you are saying. I do not disagree other than to say you still show the Axis player as being restricted in total possible points earned.

I sort of agree with that ... except that both players are in a pool of the same size. The Allied player was able to grab a bigger chunk of that pool because he really stood out, where as the German scores were flatter overall (no outliers). So if a German player had found a way to really stand out from the rest of the (German) crowd, he too could have scored much higher.

Note also that the variation is less than we saw with the current NABLA system in use - the highest Allied score is 'only' double the highest Axis score.

Other points to note: scores aren't reciprocals of each other any more. Also, everyone scores +ly with each game, which is nice Finally, it still allows for some luck (which might be good of bad, depending on your POV).

Deez,

I somewhat agree that your method of grouping players has merit. But a problem I see is that you're essentially playing two tournys. One for the Allies, and one for the Axis. At the end of the first round of play you will have found the best Axis player for that group of scens, and the best Allied player. What then? Who plays who in nthe finals, using what method(s)

Also, I'm not convinced it's an Allied/Axis split that is required. In this case it may have been, but that could also be a coincidence generated because the Allies were the attacker in 4/5 scens. Perhaps the groups need to be split not by nationality, but by role. :confused:

Jon

* Disclaimer *

I'm viewing this as an intelectual exercise, not a call to go back and re-score ROW V. Think of it as a Process of Continuous Improvement. The existing ROW/NABLA system is good. Very good. but it can be better. I think. In the examples I'm generating I'm using the just finished ROW scores not because I want them changed, but because they're available and it gives a chance to see what effect different models have on real data.

Dawg deserves to go through from Group 7 because he kicked my butt (and I was Allies against him in Malame ), not because he was somehow unfairly favoured by the system.

[Sivodsi]

Absolutely, Dawg won fair and square in our group - I've never thought overwise.

As far as I'm concerned all this discussion is over how to improve things next time, and none of it takes anything away from the superb organization and enjoyment that we've got from it so far.

[Malakovski]

Well, since my call for restraint went unheeded, I'll unheed it as well.

I think we've established fhe following things:

1) ROW V scoring is fine. Anyone who wants to rescore it is a sore loser who needs a nice, warm cup of STFU.

Everyone knew how it was going to be scored in advance, so there's no cause for complaints that it was scored that way. End of story. QED.

2) An imbalance is introduced using nabla scoring and scenarios with an average side score significantly higher or lower than 50%. The imbalnce is that one side will have a much higher maximum potential nabla score.

3) There is the possibility for the imbalance to be significant enough to affect results. Given a large enough sample, it's bound to affect some players.

4) Complete normalization of the score (the other extreme) doesn't adequately reward someone like WN who scores 58% above the average.

Some suggestions so far:

a) Normalize the scores per scenario based on possiblility. See problem 4) above. It's not fair either.

Group the sections by sides, so you're always comparing apples and apples. Problems have been discussed in previous posts--essentially two tounreys, not really "sections" anymore, probably an administrative nightmare, and what happens in the finals, when you need to compare apples and oranges again, and there aren't sections anymore to limit the effects of the imbalance?

c) Partially normalize the scores with some kind of formula, such that big wins are rewarded, but not so much that one alone can determine a section winner.

Suggestion c) was mine, but I think I have a better idea. I'm thinking it out as I type, so who knows the merits. But how about this:

d) Normalize scores across all the scenarios.

Here's what I mean. Sum the total potential by which you could have exceeded the average for each scenario, and divide by the number of scenarios.

That number will tell you if a player had an unfair advantage due to which sides he played, taking all the scenarios into account. If the average of the average scores is 50%, then overall the sides that player had were even, and imbalances cancelled each other out. If that number is lower than 50%, then the player had an overall higher potential nabla score. If higher, a lower potential.

The final, total nabla scores would then be adjusted on the basis of those numbers, to negate any cumulative disadvantage a player had due to scenario imbalance.

Here's the advantage I see in such a sytem:

It takes into account the player's total situation in the tourney. The imbalance we've discovered in nabla works boths ways, so when the system is applied to a set of scenarios, in theory it might mostly cancel itself out, or it might multiply into a really big disadvantage. With a large enough sample, you'd probably see both.

Take WN, for example. I've used his Maleme score as an example of imbalance, but this is only true if we consider that scenario in isolation.

He had the disadvantaged side for Wet Triangle (the other unbalanced scenario), and when we look at his "advantage" across all the scenarios here's what we find:

The total of averages for all the sides he played was: 72+22+48+54+49=245. The average of the averages is 49%--almost dead even. In his case, the imbalances in Maleme and Wet Triangle cancelled each other out. Cumulatively, WN had no significant advantage in total potential nabla score.

When you take the sides he had in all five scenarios into account, his potential was higher than the average by only 1%. So a theoretical final adjustment of scores to take account of overall scenario balance would harly affect his score at all.

On the other hand, the absolute worst potential someone could have had in ROW V was 72+78+52+51+55=308, averageing to 62. He could only beat the average by 38% overall.

I don't know if anyone actually drew this batch of sides, but that's definitely enough of a disadvantage to make the difference between advancing to the finals or not, especially if faced with someone who had the best possible potential.

In the theoretical match up between the luckiest and unluckiest side combinations, the player with the luckiest would have the potential to beat the average by 24% more per scenario than the unluckiest. That's a big number, though it's unlikely given the sample size that we actually had that match-up in a section of ROW V.

So, given the above analysis, I see two things.

On the one hand, I think we can see that nabla is actually pretty robust when applied to a group of scenarios, since often times the "advantage" is cancelled out.

On the other hand, the potential remains for a player to be disadvantaged, but only if he is unluckly cumulatively, and/or faced by players who were lucky cumulatively.

In summary, I think the only change needed in the nabla system is a final step in scoring which looks at each players total potential score to see if anyone suffered an overall disadvange to random scenario side assignment.

Adjusting final scores in this way would not, for example, penalize WN's huge accomplishment in scoring 90% in Maleme as the allies, since on average he did not have a high potential score, but it would also prevent a player from suffering a cumulative disadvange due to luck of the draw.

I'm sure I explained that a little ineptly, but to my mind this solves the problem with a minimum of changes to the nabla system, and fully corrects any imbalance where it counts--in total.

***

Right. Just read through this again, and I know I explained it ineptly, and some bits are terribly wordy. Bear with me. I think this sort of final score adjustment would completely address the imbalance we've noted without penaziling individual great performances.

***

Edited for clarity. Hope that helped.

A very interesting side note: the analysis above shows that WN's huge total nabla score--one that no doubt helped motivate this investigation of the nabla system--was not in any way due to luck of the draw in scenario sides.

Enormous though it is, he earned 100% of it.

Or at least 99%.

[ June 12, 2005, 06:38 PM: Message edited by: Malakovski ]

[BigDog944]

Originally posted by dieseltaylor:

And the flip side is that that score is debited to his opponent. So in Maleme WN opponent was -2.74 but then BigDog944 won all four of his remaining games and yet ends up at -0.43.

That would suggest looking at the overall list that he is not that good which seems a travesty given his subsequent victories.

Thanks, dieseltaylor! Your post has provided a salve to my wounded pride!

I hope everyone gets a chance to read my AAR for the Maleme battle. Too much hope for the Axis side rested on the airplane actually showing up, I think.

[dieseltaylor]

I thought yesterday that I should have edited to to clarify the point of Axis /Allies tourney. All sections can play either sides - provided they are all that nationality for that Battle.

So Section One is Axis battle each odd numbered battle and Allies on even numbered battles. The fact that they have all played the same side and battle in each round means nobody is disadvantaged comparative to the other players in the section.

Given 12 sections the pattern is repeated six times. You could therefore have genuine multinational playing section winners : )It is also transparently simple how it operates.

Arguably you could run it as a single round simply with the top player on each "side" being given winner status - in Group A or B. A title justly earned over 5 battles against 36 other players - though some might feel that best Axis/Allied player RoW6 might be more easily understood.

And in the interests of fair play section winners could only play the opposite side the next time they play. : )

If a final elimination was truly desired the section winners could play off in a similar fashion with the scores carried forward or not ...

[dieseltaylor]

JonS

"I sort of agree with that ... except that both players are in a pool of the same size. The Allied player was able to grab a bigger chunk of that pool because he really stood out, where as the German scores were flatter overall (no outliers). So if a German player had found a way to really stand out from the rest of the (German) crowd, he too could have scored much higher."

You mentioned the possibility that scenarios might also give a variance due to attack/defence possibilities and I agree with that observation. So whilst I appreciate your idea I still think the section system flawed if people in it do not fight the same side in the battle.

Certainly the concept of narrowing the extremes is valid as it can really screw things where a player either gets lucky through playing a rabbit in a potentially high scoring game [unlikely in RoW!], or more likely blind luck possibilities - a la Wet, or Maleme. Or most importantly as we are all aware that to score well under Nabla you are required to score better than average the daring strategy that is a gamble and goes badly wrong.

P.S can you send me your spreadsheet!

Malakovski

"In summary, I think the only change needed in the nabla system is a final step in scoring which looks at each players total potential score to see if anyone suffered an overall disadvange to random scenario side assignment."

I have concerns regarding the para:

1.I am unclear as to whether you making this remark with mixed or unmixed sections! : )

2.Most of the scenarios are attack/ defence and that the defender has less control over the course of the battle.

3.With comparatively small scenarios luck can can have a greatly enhanced effect so I think the extremes need reigning in. Good suggestions have been made.

4.I like the positive and non-reciprocal scoring suggested by JonS - I need to think about it a bit more. I am beginning to feel I should re-read the entire Nabla pre-cursor threads!!! : (

[JonS]

Originally posted by dieseltaylor:

I thought yesterday that I should have edited to to clarify the point of Axis /Allies tourney. All sections can play either sides - provided they are all that nationality for that Battle.

Yeah, I get that now. Essentially,

nationality = role

for any given scen, which covers off that concern. But it still leaves the 'two seperate tourneys' and 'what about the finals'

[dieseltaylor]

I see ! So I have to give all the answers!!!!! : )

So ... get the 12 section leaders , play three battles, against the second best player in each section matched in reverse order after each round.

Here again as they will all be playing the same role the playing field is even. And coming second in Section is worth fighting for as there will be a chance for a title - Best Loser RoWVI : ).

Seriously though I think that it does provide effectively a worthy Plate competition and keeps the simplicity of operation, and evenness of the games all in one fell swoop.

I thnk yew : )

[GreenAsJade]

A very interesting side note: the analysis above shows that WN's huge total nabla score--one that no doubt helped motivate this investigation of the nabla system--was not in any way due to luck of the draw in scenario sides.

Enormous though it is, he earned 100% of it.

Ironically, it was not so much the one very high score that caused the investigation, but rather the highly improbablye distribution Nabla Allied compared to Axis high scores.

WN will get an unbelievably high score with any system, unless it's one that has a clause like "If your initials are WN divide by 10".

Also, it's worth noting that with 5 scenarios, *every* player has a reasonable chance of being on a "disadvantaged" side to some extent. The good players make the best of this and get the high Nabla score from it...

GaJ.

[dieseltaylor]

GAJ

"WN will get an unbelievably high score with any system, unless it's one that has a clause like "If your initials are WN divide by 10".

That actually is not correct as in the Maleme scenario the maximum scored for one side was .72 being damm near 100% victory whereas WN got 2.74. As I posted before if you got the highest Axis/Allied score in each game the results could be 6.3 or 12.10.

A player could have maxed out at 6.3 or if his mix - had been 4 Axis and the one Allied game with a lower top Nabla the score would have been 6.25.

If you reverse the sides played at Maleme of the top players the differential between the revised top two changes from 4.78 to .74 which is a hell of a difference. One you can overcome the other is impossible. The Nabla sucked because like was not being compared to like and in one Group it was decisive as to who went through. : ) Reform for RoWVI

[CobaltTiger]

I'm not involved with ROW in any way so perhaps I'm way off base here, but why make it so complicated? What not just use a points system for each scenario and then total up the points? Rank each side by total score, then assign points to each player based on how they did.

That is, the two players who score highest for each side on a given scenario receive, say, 100 points. The second highest scores for each side get 93 points each, 3rd 87 points, and so on. The system scales in such a way that players who do very well are rewarded more than players who do poorly. There are 7 points between the first and second best scores while only 3 points between fourteenth and fifteenth best scores, for example.

While rewarding players who do well, a system like this is also good at rewarding consistency over one or two good scores. It is impartial to how balanced a scenario is because each side receives the same number of points. Your score in a scenario only determines how many points you get compared to other players playing the same side.

Since I can't help myself when it comes to crunching numbers, I put together an Excel sheet to see how things would have changed. In half of the groups the top 3 players remained identical, and in three groups first place changed. All three of the changes were instances where the first place player had more than +1.5 (using nabla) on Push to Maleme playing as allies, and they dropped to second place under the points system (usually by just a few points).

I'd be happy to share the spreadsheet if anyone is interested.

[CobaltTiger]

One thing I should add: I didn't take into account tied scores. That was a bit too much to get into for 3am I guess. Theoretically you would take the total points and divide them up, i.e. a tie for first between two people would be 100 + 93 = 96.5 points each. So my results would be off by a couple points here and there.

[dieseltaylor]

Nice to see people taking an interest : )

The thinking behind Nabla goes back to 2001 - a search on CMBB will reveal threads. The underlying philosophy is here:

http://www.cs.helsinki.fi/u/hurri/nabla-system/

There were probably several hundered posts over several threads and I have to admit to hesitating to look at them to see if, or why, your suggestion was mooted or rejected!

Your suggestion seems to work provided the sections, as I have suggested before, are of one nationality.

Also the fact that I could be topscore by 51 to another 20 on 50 and 15 on 49 would seem to present some problems against the player who scores 80% and is 30 points clear of the pack. Obviously extreme examples but ...... : )

[Cogust]

IIRC the initial discussion regarding the scoring system was about the scoring used in duplicate bridge where all the tables have the exact same cards. Your score are then compared with the other scores for that exact hand and you get two points for every score you beat and one point for every score that is the same as yours.

A simple way would be to implement this solution without changes to CM tournaments as it would solve a great many issues mentioned in this thread but it would not greatly reward a player with one outstanding result compared to the rest of the field.

Most of these outliers are IMO caused by a big difference in playing strength between the opponents (say one noob meeting a veteran player or something similar) and I think that it's doubtful that these outliers should be rewarded with more than a top score (2 points for each other player that is playing the same side).

[dieseltaylor]

I tend to agree with you that the outliers do seem to be over well rewarded. Though the Maleme example shows the flaw between sides maximum scores for outliers which accentuated the difference.

I think the other point besides luck,noobies, is that in effect knowing a high risk high reward strategy IS the ONLY way to be clear of the pack means some people will attempt much riskier tactics than normal.

If they work all well and good but if in the nature of things they go wrong they go drastically wrong. Between the two players this may not be a problem but when you consider this higher than normal winners score will need to be beaten by the other players in the section.

This effect I suspect is strongest in players who believe they have a chance of winning overall and those who hold game theory dear to their hearts!!

I quite like the bridge idea. But it does assume I think that the section idea has to change again. Fine. : ). The 2001 threads probably covered this also but it is nice and simple. How common equal points would be at the end of tourney I know not but suspect it could be quite high ...

For those who want to reward the exceptional results it may be difficult to swallow so perhaps the deciding factor for ties can be the percentage of total possible scores that they obtained overall. Outstanding players reap the rewards : )

[Soddball]

I demand a recount and free wine from Winecape! Never mind that I've never played in a ROW tourney, it's the principle. :mad:

[Cogust]

Tiebreaking in the bridge scoring case could be the sum of your opponents' scores (assuming that every player faces different opponents) or the result against those you're tied against (if facing the same players within a section).

In bridge OTOH you count which of the tied pairs that have the most top scores (most point on a single deal), followed by the number of 'almost top scores' and so on. That way tiebreaking would favour the players with outlier results more than those that have five results a little above average.

[Cogust]

Bridge scoring doesn't have anything to do with sections or not, I have played bridge tournaments in both formats and they work fine. The score is evalued among the entire group of players (just like it is today with the Nabla system) but the only important thing is to win your section (just as today). It is IMO much more fair to play in sections as then all your competitors will be facing the same players and that will level the playing field within the section.

[dieseltaylor]

Cogust

"It is IMO much more fair to play in sections as then all your competitors will be facing the same players and that will level the playing field within the section."

There seems to be an implicit assumption that the scenarios will balance out and that in each game both sides will have the opportunity to score equally well. As it transpired effectively all the games in RoW were primarily attack/defense. If you hold that attackers have more choices available to them to win big then the offense would be an advantage.

The better the winning players score the lower the losing players score so in the worst possible case the winner scores 36 points and the loser 0 so in that section it could be game over. The potential for winning was not available to some of the players in the section - and there are only 5 opportunities to score.

I also see that duplicate bridge is played over a minimum of 24 hands so I suspect luck/good play are less/more likely to be effective.

Bridge scoring could work effectively provided the sections are playing the same side in each battle as the performances are directly comparable. In theory the section could perform 36, 35, 33,25,19, 6 and it would be an exciting contest to win the section.The recipricocity on playing within a section with unequal battle chances could leave it as 36,35,33,0,1,3. Section ruined for some players.