The "Analysis of Nabla scoring system" thread

[Holien]

Originally posted by Nestor:

This is a fascinating thread and without really understanding the detail, it does seem that flatter would be better (perhaps because I would have finished equal 27th, rather than 28th overall )

One thing that I've been pondering is whether all scenarios should be treated equally when Nabla scores are being generated. What I mean is, shouldn't each scenario's contribution to the overall Nabla score for a player be weighted based on the number of points in the scenario?

This may be bollocks - I've been trying to make this post make sense for 10 minutes and I'm not sure I've succeeded. It just seems fairer to the total score for a player that scenarios with fewer points have proportionately less impact on the overall score, but I seem unable to express this logically!

Does anybody agree and more importantly if you do, can you express it in a more scientific fashion

What do you mean by the number of points in a scenario?

Are you effectively saying that in easier scenarios the points should be less or more?

How do you judge if a scenario has been easy or hard?

I am leaning towards this being bollocks but out of zany ideas good stuff can flow.

[Kanonier Reichmann]

I think what Nestor is, in effect, saying is that smaller scenario's should count for less compared with larger scenario's that have more units are consequently more complex & therefore should count for more for that main reason? Perhaps?

Regards

Jim R.

[Malakovski]

Originally posted by Nestor:

It just seems fairer to the total score for a player that scenarios with fewer points have proportionately less impact on the overall score, but I seem unable to express this logically!

First, let me also say that the +-10% doesn't sounds right to be.

That, along with what Nestor brings up above, is along the same lines of my earlier suggestiong to adjust based on total possible nabla score based on which sides were drawn.

All these are attempts to further compensate for overall luck of the draw, but I think the original nabla system alone may be the best mix of complexity and fairness.

While there is some merit to attempting to take account of whether a player faced, on the whole, stiff or easy competition, adding more steps to the scoring and taking more and more factors into account also has the effect of making the score more artificial.

Original nabla rewards consistency, and looking at the scores you can understand them in that light. My concern is that if, in addition, you try to handicap or adjust the scores based on other factors, the scores may become unintelligible because it won't be possible to see where it's coming from: consistency, big wins, handicapping?

I would still be interested to see if looking at total potential score has any merit, but I strongly suspect plain old nabla, version 1, is the way to go.

[GreenAsJade]

All this recalculation can't be right. My score is still negative.

GaJ

(On a serious note, I think a system where you have to calculate then recalculate would be going too far. Given that the original system (prior to losing the DOS program) appears to be doing a good job, the conclusion I would reach is that it was well thought out and should be readopted now that we have the opportunity to do so.)

[Malakovski]

Does anyone have the skill to translate the formula from the DOS program into a spreadsheet? It sounds like it's a lot of trouble to prepare the data files for processing...

[Nestor]

Kanonier Reichmann gets the prize (thanks, Jim). I was thinking of it as a separate issue to the starting point of this thread.

I'm assuming that we've got a Nabla system that deals with the issues raised by Malakovski, GaJ and others and simply asking whether a Nabla score of, for example, 0.50 for Maleme should be worth as much as a 0.50 scored in Moltke. Or should the Maleme score be multiplied by a factor (0.75, for the sake of argument) giving a revised Nabla score of 0.375, to reflect the far smaller number of points 'on the board' to be either gained or lost in achieving the raw percentage scores which Nabla works from.

This would be applied across the board, regardless of which side you were playing and different scenarios would have different multipliers, dependant on their 'size' relationship with the largest (Moltke in this case).

I'm not saying that achieving 0.50 in Maleme was 'easier' than Moltke, simply that each point scored for kills/VL's in the smaller game had a greater impact on the final % scores than in Moltke (which in itself might have contributed to the 'imbalance' of Maleme??) and perhaps some correction should be made for that when determining the overall score.

Ok, this may still be bollocks, but it now has numbers in, so it's officially more scientific. Does it make any sense?

[Malakovski]

Originally posted by Nestor:

I'm not saying that achieving 0.50 in Maleme was 'easier' than Moltke...

That's the real issue with adjustments like this. True, it's a different skill to direct a large battle than a small one, but why weight the large one more heavily? It is not necessarily any harder or easier, as you point out, just due to size.

The principle of rewarding consistency would indicate that a good mix of scenarios would include some large, some small, some offense, some defense, some armored, some infantry, but I don't see a good reason to decide that large ones should count more, any more than deciding that infantry battles should be weighted more heavily, or that offense should be given more importance.

If consistency is the virtue, I don't think we want to give any preference to one type of scenario over any other.

The special case was imbalance, but it seems that simple adjustments to the nabla formula can remedy that sufficiently.

[Holien]

Nestor I now understand what you are saying.

Yep Scientific Bollocks.

KR needs a prize for being so close every time but not quite getting there. Nice to see he has not lost his touch.

H

[GreenAsJade]

The difference between Wet and Maleme Nabla scores was nothing to do with the size (in case this what someone is saying - I'm not sure). Consideration of size as a factor in significance of scores is a whole new idea. It might be worthy of consideration, but it's nothing to do with the original issue being debated (just to be clear). IMHO a sceanrio is a scenario... each is worth the same as the next in a tourney.

However, on the original topic:

The difference between Wet and Maleme was that the Axis average was way higher in Wet, limiting the maximum score that the best Axis player could achieve.

The direction of discussion has been to explore how Nabla can counteract this by limiting/squeezing/flattening the scores so that the best Allied player can only score as much as the best Axis player.

I've come to think that this is not especially appropriate. It is certainly much harder to achieve the outstanding score as Allied Wet than to achieve the maximum score as Axis Wet. It's simply the case that the room for exellence is less in Wet for the Axis. That is WHY the average was higher.

The flattening approach is (as someone observed) a move in the direction of rewarding consitency but not recognising extreme skill when the opportunity to flex that skill presents.

I actually think this is not the best approach.

I think the real solution is to have an Axis and an Allied winner in the tournament, and only compare people who played the same scenario against each other.

Then the focus would be on using maths to take into account the difficulty of the opponents each person faced.

IE such an approach would be new - *instead* of compensating for scenario balance, compare only people who played the same scenario and compensate for the differences in opponent strength.

Note that such a new approach *removes* one source of "unfair" comparisons (unbalanced scenarios) and allows the issue of differening opponent strength to be cleanly tackled.

Nabla doesn't even start to address the issue of opponent strength.

Summary: for me the outcome of this discussion is that the original Nabla method was OK in what it tried to achieve, but we've learned a lot about its flaws, and seen that there is a completely different approach that could be much fairer in rewarding excellence of play.

GaJ

[ June 17, 2005, 08:58 PM: Message edited by: GreenAsJade ]

[Treeburst155]

...been gone for a week, and the computer I was counting on at my destination burned up.

The use of a fairly flat curve addresses the problem of any single scenario having too much influence on a player's final tourney score. Nabla writes at some length on this subject on his site.

The problem of scoring opportunity due to scenario balance would be solved by the Axis/Allies approach spelled out earlier. We are left only with the "opponent strength" issue, which is important IMO. If I score the median for my side against a hotshot, I would like to see my performance compare favorably to someone who scores the median against a newbie. The scoring process I outlined above would accomplish this.

Malakovski writes, "While there is some merit to attempting to take account of whether a player faced, on the whole, stiff or easy competition, adding more steps to the scoring and taking more and more factors into account also has the effect of making the score more artificial.

Original nabla rewards consistency, and looking at the scores you can understand them in that light. My concern is that if, in addition, you try to handicap or adjust the scores based on other factors, the scores may become unintelligible because it won't be possible to see where it's coming from: consistency, big wins, handicapping?

I would still be interested to see if looking at total potential score has any merit, but I strongly suspect plain old nabla, version 1, is the way to go.

Right now, we have the final tourney scores before the AAR adjustment, and again after this adjustment. Similarly, we could publish final Nabla scores before and after handicapping. This would make it easier for player's to evaluate their performance in the tourney, making the scores more intelligible.

Moving on to Malakovski's idea of "total potential score", I can't see how this can be incorporated into Nabla. With the Axis/Allies idea, there would be no need for this anyway. All a player's competition would play the same side in all the scenarios. "Potential score" would be the same.

I'm going to begin testing various methods of handicapping. Perhaps something as simple as taking the average final tourney score of all a player's opponents and adjusting the player's final score appropriately would work. The exact adjustment method needs to be carefully worked out. I'll experiment with the ROW V results and post handicapped results here, along with the calculations used to achieve them.

If handicapping does not appeal to players, I would suggest that groups be organized based on previous tourney performance. The hotshots play the hotshots, etc. with tourney newbies having their own groups as far as is possible.

[Treeburst155]

After working with this handicapping idea, I've come to the conclusion I don't like it. An exceptional player tends to cause lower final Nabla scores for those in his group. Any handicapping based on these lower scores comes back to bite the exceptional player who caused the low scores in the first place.

Also, a computer program would have to be written to perform all the handicapping arithmetic. It's way too big of a job to do manually, not to mention the high probability of errors.

Finally, the flat curve keeps a lopsided matchup (hotshot vs newbie) from having too much influence on final Nabla scores.

I think assignment to groups based on prior performance is the way to go. It would take someone like Nabla to work out a good handicapping system IMO.

[WineCape]

Thank you Treeburst - knew that you would chime in here sooner or later, being aware what Nabla can and cannot do.

[Treeburst155]

Hi WineCape! I was wondering if you were following this thread.

Handicapping, an idea I love, would only work if it was based on PRIOR tourney performance. This way a hotshot doesn't get bit by his own great performances unless his victims from the previous tourney just happened to be assigned to his group again.

Changing the subject a bit, using a more competitive curve (flatter) than was used in ROW V should create more incentive to produce AARs. The gaps between players will be much less, thus making the AAR bonus more attractive.