Jump to content

Scenario Depot Blues


Recommended Posts

Once upon a time you could go to the scenario depot and be assured of finding a quality scenario quickly.

If you wanted to experiment with something unknown, you could, but you didn't have to.

If you wanted a sure-fire good scenario, you picked on with a high review rating and away you went. Sure, we may not all have agreed whether one scenario was really 0.26 better than another, but who really cared?

Now it's a lucky dip. You have to either just take one who's name appeals to you, or trawl through all the reviews trying to make out if it will be OK.

So sad :(

[ October 22, 2004, 12:12 AM: Message edited by: GreenAsJade ]

Link to comment
Share on other sites

  • Replies 113
  • Created
  • Last Reply

Top Posters In This Topic

Me too - as I said at the time.

The reason I'm raising it now is that we've had some time to try the new arrangements, and I, for one, find them much less helpful than before. Its still nice to have a place where the scenarios are kept, but it was a way nicer facility before.

We've lost ease of use to solve a few arguments between scenario designers about rating systems :(

For me, I'd say "who cares what the system is, just give us a least some quick way to sort out the scenarios that people liked from the ones they didn't"!

Link to comment
Share on other sites

Come on, you whiners!!! You make it sound like it's the most complicated and time consuming process in the world to open the scenario description and check the reviews. How much time can it take? :mad:

Besides, shouldn't you really be playing those with NO reviews, so that you can write one!? ;)

[ October 22, 2004, 05:44 AM: Message edited by: Sergei ]

Link to comment
Share on other sites

Hmmm....

Bless Admiral Keith for all his hard work. The scenario depot is a gold mine.

OK, I have one scenario up there. The 2 reviews it has come under historical reviews so for whatever reason '? :( ?' they don't count..?

So for what ever my opinion matters I think an 'improvement' as such, isn't always for the better. I really did like the old system but like Sergei says, a review shouldn't bias your playing choice and navigation is a little easier.

Link to comment
Share on other sites

To paraphrase the famous quote, "You can't please everyone all the time".

This has been covered in threads long gone, but the predominant reason the old cumulative rating system was scrapped was that it was supremely subject to "ballot box stuffing", abuse, and whining. Players were intentionally submitting either overly high or overly low reviews to skew the cumulative score. Authors were perpetually complaining that players were posting low review scores which adversely affected their overall high rating.

This made the entire cumulative rating system invalid. Now each review stands on its own merits, and contributes nothing to an overall score.

If anyone can design a system where players can post scores which accumulate to an overall rating, which also cannot be maliciously manipulated, I'm open to ideas. I'm skeptical that it can be built, simply due to the fact that if a player truly hated a scenario and posted a low score which devalued the overall score, the author will claim "trolling" and petition to have the review removed.

Prior to The Scenario Depot v2 redesign, I petitioned for suggestions from players and authors on what features they would like to see implemented. Pretty much everything that was capable of being coded was implemented.

Maybe now is the time to begin taking in ideas on what people would like to see in The Scenario Depot v3. If you preferred the old system, then help design the new system. If you want your suggestion to have any merit, you must consider the following:

* How to prevent abuse by both players (reviewers) and authors.

* What to do with the reviews from the old systems. Incorporate or segregate?

* How to rank various scenarios against each other. For example, should historical scenarios be ranked in the same categories as fictional?

* Now that authors can define their own rating labels, how will those scenarios be ranked? Perhaps have 2 to 4 hard-coded categories, such as Play vs. AI, then allow perhaps 2 to 4 additional author-defined categories.

* Authors may selectively choose to have numeric, textual, both numeric and textual, or no reviews at all. What to do about scenarios which have no reviews, or simply textual reviews? Perhaps how the previous problem is handled will define how this one is solved.

* A hard-and-fast set of rules by which players and authors alike agree that reviews can be stricken from the overall score. Perhaps something similar to a flag which can be set by the author, which publicly identifies the review as a possible troll-attempt, allows the poster to respond, and removes all values from the cumulative score. Hmmm, actually not a bad idea.

I have no problem re-instituting the previous system, but prior to doing so, there MUST be a viable solution to the previous problems.

Now is the time for everyone to weigh in on this. I'll potentially be out of work in February, so this is a good time to begin putting together the specifications for this. No idea will be refused, but those ideas which actual include some methodology (coding, math, logic) will be given more weight.

=====================

@jwxspoon - Please clarify your "1 for 1" idea. How can that be enforced?

Link to comment
Share on other sites

Originally posted by Admiral Keth:

To paraphrase the famous quote, "You can't please everyone all the time".

* Authors may selectively choose to have numeric, textual, both numeric and textual, or no reviews at all.

* A hard-and-fast set of rules by which players and authors alike agree that reviews can be stricken from the overall score. Perhaps something similar to a flag which can be set by the author, which publicly identifies the review as a possible troll-attempt, allows the poster to respond, and removes all values from the cumulative score. Hmmm, actually not a bad idea.

True, and both good ideas!

If a score seems inconsistent and a designer wishes the score to be striken from the record (but a review should always stand, as trolling is usually pretty obvious and will weigh against a reviewer) then let it be noted.

Perhaps you could have a continuous scoring system for the reviewer. A conglomerate score that represents their reviewing history, much like with ebay's feedback with the number of scenarios reviewed and an agregate score...

Hmmm....

or we could just have the 'ENFORCER' to deal with the trolls

Link to comment
Share on other sites

Originally posted by Richie:

...or we could just have the 'ENFORCER' to deal with the trolls

Although a machine gun may be a bit much, perhaps a dedicated team of "Review Reviewers" might keep things in line.

Steve Overton and his group have been instrumental and extremely helpful in several areas, including patrolling The Scenario Depot for various reasons, but I'd rather they spent the bulk of their effort making more scenarios.

I'm now envisioning two seperate flags for reviews: troll and hidden. The Troll flag discounts the associated rating from the cumulative score, and emails the reviewer that the author has flagged the review, but the review still remains publicly visible. The Hidden flag removes the review ratings from the cumulative score, hides the review, and emails the reviewer.

At such time that the new system is implemented, I can forsee one or two "Moderator" positions occurring, which would have the capability of touching reviews, and flagging them as Troll or Hidden, along with emailing the reviewer. The email FROM address sent to the reviewer will be that of the person doing the flagging, so that the reviewer can respond to the correct person.

Link to comment
Share on other sites

How much is 'trolling' a real problem, and how much is it of that people have very different mind sets, experiences etc.? I think there are people who give scenarios weird ratings with odd explanation, but I don't think all of them really are trolls. Just... different.

Maybe you could have the possibility of rating ratings, maybe, in a "was this review useful?" way. If a review has a certain amount of negative ratings, then it would have diminished or no weight in the overall score. Designer couldn't vote (but he can of course leave his response and ask to explain if the review was strange) to avoid him from purposely downgrading the value of negative reviews no matter how reasoned the opinion...

It sounds unnecessarily complex, though. And then of course you'll have to develop measures to prevent trolling the review rating system... :D:D:D

Link to comment
Share on other sites

OK, forgive me, I've had issues uploading to The Depot in the past but that was entirely my own making and no reflection on the excellent website and resource you run. Please don't get me wrong, I'm trying to be constructive.

My inexperience is my own fault...

a) This is a feature available on initial upload or a feature available at any time with the current version?

For example, the 2 reviews I have for my Reichstag battle can be deleted or made concurrent?

B) Yes, the reviewer grades, for example, 3 scenarios at 1.5, 5 and 3.5 for a cummulative reviewer score of 3.3... a bit like I have 355 posts, at the mo'.

c) I think a review should stand regardless but with no score. Flagged as score removed by the designer. With an average reviewer score and the review itself, readers should be able to detect trolling, but a moderator would certainly help.

Now there's alot of commitment!

[ October 22, 2004, 09:03 AM: Message edited by: Richie ]

Link to comment
Share on other sites

Admiral - what would be neat wouuld be to have a "point system" awarded for scenario reviews. A new registrant at the Scenario Depot might start with 10 points in his 'account'. As he downloads new scenarios, his account balance is reduced. When he posts reviews, his acount balance is credited. Etc.

Kingfish - You bastard. Why did I ever post some of those old army pics?

jw

Link to comment
Share on other sites

Originally posted by Richie:

a) This is a feature available on initial upload or a feature available at any time with the current version?

The author may change the labels associated with the numeric values at any time.

Originally posted by Richie:

For example, the 2 reviews I have for my Reichstag battle can be deleted or made concurrent?

This confused me.

Originally posted by Richie:

B) Yes, the reviewer grades, for example, 3 scenarios at 1.5, 5 and 3.5 for a cummulative reviewer score of 3.3... a bit like I have 355 posts, at the mo'.

OK, that appears to be a viable method by which reviewers can be "self-rated". However, what happens when they intentionally post high scores all of the time? They are effectively invalidating the system by abusing it.

Originally posted by Richie:

c) I think a review should stand regardless but with no score. Flagged as score removed by the designer. With an average reviewer score and the review itself, readers should be able to detect trolling, but a moderator would certainly help.

Now there's alot of commitment!

OK, looks like one vote for the Troll flag.
Link to comment
Share on other sites

Originally posted by jwxspoon:

Admiral - what would be neat wouuld be to have a "point system" awarded for scenario reviews. A new registrant at the Scenario Depot might start with 10 points in his 'account'. As he downloads new scenarios, his account balance is reduced. When he posts reviews, his acount balance is credited. Etc.

jw

OK, let me paraphrase to see if I understand...

In effect, everyone starts out with a 10 point credit. 1 download = 1 credit decremented. 1 review = 1 credit incremented.

I like this idea, but the real test will be from the authors. Once the player has consumed his/her 10 credits, they could potentially sign up again (although they would have to use a different valid email address). In addition, will this discourage players from downloading scenarios?

Authors must weigh in on this subject before it goes on the Giant SDv3 To Do List.

Link to comment
Share on other sites

It makes me crazy when the designer states not for play as axis vs ai and the reviewer says he played as axis vs ai and the scenario sucked.

No, just a Slaphead at work...

DO'H!

But the designer should be able to therefore discount the score. I remember Rune bringing this up once...

Link to comment
Share on other sites

Originally posted by Sergei:

How much is 'trolling' a real problem, and how much is it of that people have very different mind sets, experiences etc.? I think there are people who give scenarios weird ratings with odd explanation, but I don't think all of them really are trolls. Just... different.

There have been several blatant trolls in the past, but they wer handled quickly and quietly. This was one of the reasons for implementing the Banning feature now in place at The Scenario Depot.

Originally posted by Sergei:

Maybe you could have the possibility of rating ratings, maybe, in a "was this review useful?" way. If a review has a certain amount of negative ratings, then it would have diminished or no weight in the overall score. Designer couldn't vote (but he can of course leave his response and ask to explain if the review was strange) to avoid him from purposely downgrading the value of negative reviews no matter how reasoned the opinion...

I love this idea. However, how will positive ratings affect negative ratings. As an example, does 3 negatives and 5 positives yield a +2 overall Reviewer Rating (RR)? Perhaps as a percentage...1 positive RR = 100% of the score applied to the overall scenario cumulative rating. Then 2 postitive RR's = 100% of the score applied to the overall scenario cumulative rating. A successive negative RR will result in 50% of the score applied to the cumulative rating. Therafter a successive positive will result in 66% of the score applied to the cumulative rating. One negative RR will simply eliminate one positive RR.

The higher the number of positive RR's, the less impact any single negative RR will have on the individual review ratings, thus on the overall scenario cumulative score.

Complex, and I'd need to see what effect it would have on server load, but entirely possible.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...