Atlas' scenario design discussion.

[Andreas]

Keth, I guess it is time to again link it in the sigs...

First of all - you are the brainy one, which is why you do the depot, and I just do random scenarios I have no idea how coding works, except that in the case of your site it is certainly very complex. I understand that a blank slate approach would not work due to the need to preserve the historic ratings.

Some ideas I had for a while:

- weigh the replayability score at '0', since most battles I come across (and all I do) are not designed with it in mind, and in some cases it is actually impossible to achieve that aim, because it will restrict the use of features in the scenario (how often are you going to be surprised by that 88 over there? Once FOW is gone, it is gone for good).

- Have an overall score (additional to the current average) that just shows what you thought about it. A bit like some reviewers do now in the written part of the review).

Just some thoughts - as I said, I think the Depot is a great service to the community and the designers, and you have gone far beyond the call of duty in setting it up and running it.

Regarding the idea that you force people who submit to review - I think it is unworkable, and undesirable. I only have a limited amount of time for CMBB. I use that on CMMC and for designing scenarios. I usually play little, and I could not fulfill that request at all, especially since many of my games are for testing other peoples work-in-progress, making reviews impossible.

I could see if I can get around drafting a rewiew guideline document, and I would put that up here for consultation. I certainly would not want to do it myself, because of my biases when it comes to design.

So, thanks again for all the help over the last year or so. Keep it up mate.

[Atlas_TH]

Keth,

Thanks for you efforts on our behalf. Any suggestions we make are not meant to detract from the excellent job you and yours have made in creating/upkeeping the SD. Your site is the best already.

Question: Can't you already zero-rate every category without hurting the overall?

Question: How many hours would it take to change the "overall" score to an independent variable?

Comment: The five star rating system could use s 1 to 5 scale and not literally "stars", per se. Not sure if that is useful enough to consider. Does that make it easier to program?

-Cheers

[Vergeltungswaffe]

Admiral Keth, for the last two days, I have been unable to review any of the CMBB operations. You may be well aware of this, but just thought I would mention it, in case.

[Admiral Keth]

Originally posted by Atlas_TH:

Keth,

Thanks for you efforts on our behalf. Any suggestions we make are not meant to detract from the excellent job you and yours have made in creating/upkeeping the SD. Your site is the best already.

Question: Can't you already zero-rate every category without hurting the overall?

Question: How many hours would it take to change the "overall" score to an independent variable?

Comment: The five star rating system could use s 1 to 5 scale and not literally "stars", per se. Not sure if that is useful enough to consider. Does that make it easier to program?

-Cheers

Zero ratings are not included in the total ratings. If a reviewer does not have an opinion regarding a particular aspect, such as PBEM, he/she should use zero.

I opted to have the overall score calced each time, as there are occassions when reviewers ask me to go back and change an erroneous value in a review. This eliminates me having to go back through and manually calculate total values for a particular scenario. To answer your question...

CMBB Only:

1) 10+ hours to recode and test the pages.

2) 2-6 hours to hand calc and validate the values in the existing scenario reviews.

CMBO Only:

1) 10+ hours to recode and test the pages.

2) 50+ hours to hand calc and validate the values in the existing scenario reviews.

I'm padding the estimates a little, but they are fairly close.

Regarding the rating system - a 0, 1-10 system was opted for so that there was a much finer degree of differentiation between the hundreds of CMBO scenarios. Sort of like using a d20 for gaming instead of a d10.

Originally posted by Vergeltungswaffe:

Admiral Keth, for the last two days, I have been unable to review any of the CMBB operations. You may be well aware of this, but just thought I would mention it, in case.

Yep, one of the many things that need to be addressed, not to mention a brutal work schedule and real-life issues (had a close relative pass away recently).

Non-the-nonce, work progresses every day at The Scenario Depot. Suggestions, ideas, comments, and criticisms always taken in the light that they are given.

[Vergeltungswaffe]

AK, hope that didn't sound like a complaint, because it wasn't. Just trying to be helpful.

I am totally in awe of what you do at the Depot.

[Warlord69]

Thanks again Keith for your wonderful service you provide the entire community.

[Admiral Keth]

Originally posted by Vergeltungswaffe:

AK, hope that didn't sound like a complaint, because it wasn't. Just trying to be helpful.

I am totally in awe of what you do at the Depot.

I didn't take it as a complaint, but I am genuinely interested in what aspects of The Scenario Depot works/doesn't work for people. How else can I make it better?

Originally posted by Warlord69:

Thanks again Keith for your wonderful service you provide the entire community.

It's just my way of giving back to the CM community. I wouldn't be doing it if I didn't believe in it. Instead I'd probably be brewing up another batch of my famous "Ole T-34 Motor Oil" Russian Imperial Stout. I suppose CM is doing it's part in keeping me from being a drunken layabout.

[Potvin]

Does anyone design CM:BO scenarios anymore? :confused:

[xerxes]

One part of the depot's rating system that I don't like is the use of "0" for "not applicable". Raters have given "5's" instead of "0's" because they dont' want to mess up a rating. Just changing the first pulldown option from "0" to "no rating" or "not applicable" would help.

As always, thanks Keith, the Depot is a great resource.

[Eden Smallwood]

Originally posted by SurlyBen:

Eden, this is the idea behind The Scenario Depot:

Yes it is, yes it is... But you realize, of course, that the relevant point which that fact doesn't change is that you had to tell me that personally!

It might not be the easiest site to use

It might not be the best site to advance your collective agenda, therefore.

(I find the new buttons baffling... It took me forever to realize that the text for each button is in the headline, and changes on mouseover... Alt text would be helpful)

Yes, me too.

Haunted Hill is listed, for example (and unreviewed)

Well it's reviewed now, but don't worry, your hint is not lost and I've long ago promised my superego to go out there and review all the non-CD ones for which I don't have a completely dismal experience.

I've posted sorta kinda one so far- Red Parachutes. Halfway through I realized that the "Replayability" totally depends in that scenario, on whether the scouts that come will come from the same exact place each time... If they do, it's only truly great once; if they can come from completely different points on the map, then it's a truly replayable one.

I haven't found a way to edit that review, it raises the question of what happens if I review it again- will it wipe out the first one, (good), or add a second one? Either way, I guess it means I should hold off on a review until I have complete facts, which would usually mean playing it twice, which means more delay and forget, which means... Anyhow,

Eden

[Eden Smallwood]

Originally posted by Atlas_TH:

I agree that designer's reviewing other designer's scenarios can be problematic, if not downright incestuous. Designers - especially ones in the same group - may choose to grade upward to ensure others do likewise for them. A good reason to only grade designs of people you do not know, which is tough in such a small community.

Pshaw.

I guess I can't say we're all adults here (someone claimed to be 17 the other day), but how's this- we're not dense. Sure, we can see understand that vested interest situation there... so what? So if a designer wants to review another designer's scenario, he could just begin the review with, "the following review is by a fellow designer".

Good enough! If its so difficult to get reviews from Joe citizen then why be so strict with the whole thing? The numbers ratings are kind of silly anyhow, it's what the person says that is of value, imho.

I could have reviewed a scenario the other day which I would have given a 10 briefing, but was otherwise unplayable. So it has an overall rating of... 10?!? Ridiculous, right?

Eden

[Eden Smallwood]

Originally posted by Dook:

As someone who recently posted his first scenario at the Depot and has received only one review despite 200+ downloads (and that from a beta tester!),

I have contributed a +2 "downloads" which are in fact completely bogus. I liked the look of scenario "X", and clicked on the download. It went to Tom's HQ, the front page, asking me whether I preferred English or did I want to try my chances with German for once.

I said "Aw, heck- where is it now? I know he's got them categorized by year... what year was it...?" ( Then, for some reason, the same exact thing happened again. ) It was late, I said to heck with it and went and got another one.

putting reinforcements in plain view of the enemy.

Bad. Don't like. How does that reflect on my ability as a commander?? It doesn't- it's just *there*. Maybe they get toasted, maybe they don't.

Even worse is when reinf pop up already under fire, *sitting* on a road! Aargh!

In a similar vein, for me, my forces should not be in contact on the first turn- the presumption is that the meeting begins... well, before contact, right?!?

Now to all those designers who are thinking, "My scenario doesn't do that", I would like to say, "Have you tried your scenario with the AI *free to place units*? (ahaaaaa!)"

One more thing which I felt didn't really reflect on my abilities in a scenario- that enemy *plane*. I was playing a scenario, I have no anti air ability, my tanks are kicking butt... then all of a sudden some plane comes by and takes out a tank.

Well... ok- I know it's supposed to be historically accurate and all that. But look- the net effect of that plane was just like rolling the dice- maybe you lose a tank, maybe you don't. I dunno, but it just seemed kind of... arbitrary.

The idea of putting a link to the scenario reviews into the actual game is a great idea, but I have no idea whether it is easy, hard, or impossible.

Well I do have an idea, and it would be extremely easy, or else the programmers have arranged themselves in a completely terrible way, which I can't believe given their accomplishments. Ciao,

Eden

[Stoffel]

I think Andreas is right on the review issue.

I have created a few battles for CMBO.

Several have about 200 to 400 downloads.

Some have a few reviews but except for one they are all praising the battles.

That one review told me more about the scenario to improve it and make it better than all the others,this is all what we are asking.

Not a lot of your time or a huge commitment but a simple I like it or even better I dont like this or that but with a good reason why.

Critics can make a designer better in creating battles ,and it doesnt take long to write your opinion.

And if a designer gets better ,you get more fun in playing.

As I always have reviewed scenarios I have played,so do I think others can do that too.

Henk

[Eden Smallwood]

Originally posted by wwb_99:

Eden: you might be a noob, but you did play the battle and are hence quite qualified to comment on it. A bad review is better than no review IMHO, presuming you dont just say it sucks without saying why.

Well I'll try to remember that for scenarios by that wwb_99 guy, but I won't assume that for anyone else.

I played a scenario which I couldn't possibly in my wildest dreams imagine winning; it makes Jaegermeister look like a cakewalk. Directive #3 I could at least *imagine* winning... somehow.

But I see reviews of that scenario posted later, like "Hey- cool scenario dude. Nice little fight. Sweet aftertaste, with a fruity bouquet and good balance." What am I going to say then?!? Ha!

Eden

[Eden Smallwood]

Originally posted by Admiral Keth: </font><blockquote>quote:</font><hr />Originally posted by Me:

For #3 and #4, maybe you guys could all get together and make one official central link website database thingie for all these scenarios. Even if they're hosted in a bazillion different places, at least have one d*mn site which points to them all, is well organized, has a nice little review thingie, let's the user view them in different arrangements, (by newest, by medium, by Allied vs AI, you know). Well dang that would help I think. The way it is now... Yuk.

How is this different than what The Scenario Depot offers now? Designers can host their own scenarios on their own sites, if they wish, or send them to me to host. How can The Scenario Depot be better organized? You can sort alphbetically, by Newest, or perform a search on just about any field in the Synopsis. I'd appreciate a better idea of what you mean by Yuk.</font>

[Potvin]

For those designers who have an interest, may I suggest you pick up an old copy of the computer game, "East Front II."

More of a divisional-sized representation, it has a few good battles that may inspire you.

-Atlas_TH

[rune]

While i think the job admiral keth and big dog do is above and beyond the call of duty, i have never rated a scenario there, and take rating numbers with a grain of salt.

I do not post reviews, since as a designer, i know what I like, may not be what another designer likes. Matter of fact, seldom are tastes the same.

Which brings about the scores. Let's look at jaegermeister, the briefing is a 3 or it is a 8 or 9. OK. The format is the same as all the other briefings on the CD [done for a specific reason], yet is marked worse?

Another scenario was a tutorial, but marked down because it isn't replayable. The scenario is pretty much historical, but I marked it semi-historical. I am told to add more anti-tank guns to the Russian side. Ummm...that would destroy the whole point of the tutorial/scenario, as well as destroy the historical battle.

I use the feedback as a guide, and that is all. People bring their own thoughts and prejudices to a review, if they realize it or not. How many of us have gotten a "I lost, so the scenario is unbalanced" when the next review says balance is fine?

Replayability...well the engine does NOT allow for vsrious starting points, just random times the forces can show. Replayability is limited in EVERY scenario.

All that being said, I love reading the comments and feedback I get. I just take it with a grain of salt. I actually get more feedback via email then I do on the depot. However, I THANK Admiral Keth and Big Dog for what they do, because I know a lot of people got my scenarios that would not have gotten them any other way. Most people don't bother reviewing....a shame really, but understandable.

Last point: I differ from a lot of opinions here. I think the scenario should be played with default locations the first time through. I know I place my units historically if I can, or for a reason if not... I don't lock them however, so the next time the scenario is played, some randomness comes into play.

I was lucky enough to learn from some of the best, and I got to work with some of the best for CMBB. It was a pleasure working with the lot of you, and seeing different styles, different feedback, etc. Yep, you can blame me for what went on the CD, it was basically my call. I did ask, fought for, and finally got on more scenarios then was originally planned for. I thank each and every scenario designer that worked on the scenarios for the CD, as well as thanking those others that just keep the game going by making new scenarios/operations.

Rune

[Potvin]

Rune,

I hear ya! LOL.

Very true re: no two reviewers are alike in their preferences.

A few examples:

1. I hate urban fighting. Anything that happens in a factory makes me drool with boredom. I tried to review your "...Factory" scenario, but my bias got in the way.

2. I like big, big maps. The whole idea behind German tactical, operational and strategic planning was "freedom of movement." Even if it takes five or ten turns for the action to start, I like the control to attack or defended in a greater area. (From what I read, MOST people like the action to start right away.)

3. I prefer PBEM and head-to-head. AI scenarios have ZERO replayability for me. In fact, if CM:BO and BB had no pvp capability I would never have bought the games in the first place.

4. ...etc...way too many preferences to list here, which is the point, I think.

Anyway, I think the "DISK" scenario designers did a fairly good job. IMO, there are some truly mediocre scenarios, there are some great ones and there are a few lousy, lousy ones that I cannot believe made the grade. (Rushed into production?) There are many new scenarios at the SD that are much better.

That said, I think the disk "designers" did a good job presenting a diversity of battles.

Ask someone else, they'll disagree...

[rune]

Exactly my point. Look at my tractor works. The map got 5s from two reviewers. However, the buildings were researched, and are the EXACT layout they were in real life. Berli and I got maps and aerial photos, and i actually measured the size of the buildings. If you look, my scenario is a small version of Berli's to the volga operation which is huge. However, the map is average? Sorry, those damn Russians should have made a better factory.

The scenarios were not rushed. But for each scenario you think is lousy, someone else will like it. That is my point...scenario reviews are subjective. However, I still thank Admiral Keth and Bigdog. One of my scenarios was downloaded 501 times with 1 review. At least i know 500 people got the scenario. Without the two of them, a lot of scenario would go un-noticed. So, I thank them and all the hard work they put in.

Rune

[Stoffel]

Rune you made a good point here.

I have that problem too,2 scenarios downloaded 476 and 200 times and 1 review.

I dont care for the grades given,I only want feedback and-or good ideas that can make a scenario better.

[xerxes]

Reviews are by their nature very subjective. There were scenarios in CMBO that most people thought were great that I didn't care for. I also think some scenarios are very underappreciated. A lot of it depends on what the player looks for in a scenario that makes it fun for them. Some people prefer scenarios that play like QB MEs, personally, I don't care for this style at all. It's really a matter of taste.

It would be interesting to see a list of all scenarios rated by a reviewer. It's like a food critic, some critics have your taste, others don't.

[Tarquelne]

First, AK, thanks for for the Scenario Depot - great site!

Zero ratings are not included in the total ratings. If a reviewer does not have an opinion regarding a particular aspect, such as PBEM, he/she should use zero.

I suggest that be put in VERY LARGE PRINT somewhere.

feedback

That word's come up several times in the last few posts, and it was one I was thinking since ES's first post on the topic: I second ES's points on why players often don't write reviews. Luckily, it sounds like what many designers really want is just some feedback. Hopefully a compliment, but constructive criticism is always welcome, correct?

What if, in addition to "Review this scenario" you could simply "Leave feedback for author" No ratings asked, just a small text box. Or, probably eaiser - change "Review this scenario" to something like "Give feedback NOW, sucker!", "Review or be damned!" or "As if we cared, what's your opinion, noobie?"... or something like that.

And make the link bigger, why not?

And, while I'm on the subject of potentially offensive technical fixes, why not have the site track what scenarios you download, and then ask for reviews when you go back for more?

Because it's a lot of work, it's irritating and it's intrusive you say?

Well, yeah, I can see that. But I don't know just how much work it'd be for AK, and maybe he could "soft sell it." SD is awefully well put together, I wouldn't be suprised if AK could pull it off... perhaps a fairly narrow frame off to the right titled "Review me?" with a list of unreviewed scnearios? That area's empty when I'm at the SD. I don't think that'd feel much like an EA registration reminder.

Oh yeah... I, too, will be better about at least leaving some feedback, if not an actual "review".

[Ken Talley]

Very interesting thread.

I've noticed on the player v. player message boards that the games are most often QBs, which got me to wondering: what are we doing wrong? I think players want more control that we can give them. They want their units, their time-line,...their game.

QB's are OK but the maps just don't compare to a well done, hand made map. I do believe/hope we will see more demand for QB's with custom maps.

I think scenarios can be designed with plenty of replayability for head-to-head matches through the proper use of terrain and setup zones. A good hand designed scenario should be far more enjoyable than a QB. The one drawback is both players have to have the same scenario. It is a little thing but is a drawback for TCP/IP games. Although I suspect a lot of people do like creating their own task forces in QB's. I think that is the major advantage of QB's vs scenarios.

As to ratings, I noticed that of my scenarios, two of the three ratings seem to focus on the quality of the game vs the AI. I never design my scenarios specifically for play against the AI. The strat AI just simply can't put up a good battle. It has great difficulty with using reinforcements and exit zones or dealing with more than one flag. My scenarios can be played against the AI, of course. However a rating based on an AI game is not going to accurately portray the quality or deficiencies of the scenario as intended to be played.

PS: One thing that is funny. I have received probably 10-12 email comments on my scenarios vs the 3 official reviews. Getting the emails is always nice.

[ November 02, 2002, 02:53 PM: Message edited by: Ken Talley ]

[Ken Talley]

I just took a quick look at the scenario depot.

When I choose a scenario to play, I use several criteria more than others-the date, the number of turns, and the quantity of troops(huge, small, medium).

That information is not available from the selection screen. We do get the troops types and reveiw ratings which is very useful. But to get turns, size and date, we have to open each scenario and dig for the info.

I don't know how difficult it would be to add that type of information on the scenario selection screen but it would be a nice additional filter.

[tracer]

To Rune and Andras, and all the other people who are making scenarios, my say probably does'nt count for much but, I think you guys do a great job on the scenarios. Heck, rune, I really liked the clash of titans, and I do review the scenarios at the depot. I think it is a matter of taste on what kind of battle or OP each person likes, and if some one does'nt like the work you guys do FOR FREE, heck there's a editor that comes with the game let em make thier own. And one more thing, people seem to forget that a lot of you guys including modders are doing this in your spare time and it is FREE, so won't people give constructive critism that can help people make this a even better game then what it already is, and stop down grading so much. I'm not saying that there is many that do, most people on this forum don't but to the ones that do remember this, these mod sites and scenario sites are FREE.