The future of user made scenarios – and the lack of community feedback

[umlaut]

I´m starting this thread in the hope of opening a debate about the lack of feedback to scenario designers. This is a debate that has been popping up at intervals in threads with other subjects, but to avoid hijacking these threads I´ve decided to start a separate thread on the subject (Even though my previous scenarios have all been CMBN, I´ll post it in this section, because this is the most active part of the forum and probably where most new scenarios will be published)

I´ll try to outline the problem as I see it, briefly describe the work involved with releasing a scenario, possible reasons for the lack of feedback and possible solutions. I very much hope other scenario designers will chime in with their own views and suggestions.

The problem:

A quite regular occurance on these forums is some CM player complaining that there are too few user made scenarios available. This has some times provoked me to reply along these lines:

Dear complainer, you´ve got no-one but yourself to blame for that: If you can´t be bothered to give the scenario designers feedback on their hard work - then they will stop releasing scenarios to the community.

Judging from comments in other threads it is my clear impression that other scenario designers share my frustration about the lack of community feedback. If I felt I was the only one I wouldn´t bother starting this debate. I´d probably just stop releasing scenarios. Still, I can of course only speak for myself and thus I´ll use myself and my own experiences with scenario design when I try to describe the problems as I see them.

Here´s the situation from my perspective:

I really enjoy creating scenarios. Mainly because I enjoy making maps that look realistic, but also because I like to play a scenario where I get to choose the forces and the scenario´s challenges myself. Sometimes I dream up a situation and then make a map that suits it. Most of the times I make a map and then make a scenario afterwards.

It has often been said in these discussions that "you should only make scenarios to please yourself, not the community". That is of course entirely true – and as you can see, this is what I do myself.

But sometimes I decide I´d like to share one of my scenarios with the CM community. And this is where the trouble starts.

Because there is a crucial difference between making a scenario that is playable for the designer himself and making one that is playable for everyone in the community, ie one that is ready for release.

In my experience the time needed to make a scenario ready for release equals the time used on creating the scenario itself.

Making a scenario ready for release means:

- writing a briefing that is clear and hopefully entertaing to the players. Two briefings, if it is a H2H scenario.

- making a correct list of the forces involved

- making a strategic map BMP

- making a tactical map BMP

- making a preview BMP

- making a operational map BMP

- writing designer´s notes

But the most time consuming aspect above all is playtesting the scenario. This involves activating several volunteer testers, who play the scenario while they send me save files, during action reports (DAR´s), comments and suggestions. Then, when the testers have finished playtesting, I begin altering and tweaking the forces, map, victory points or AI plans based on their experiences and suggestions. After this I usually get one or two new testers to playtest the new version of the scenario to make sure there are no oversights.

All in all this process takes at least one month – usually several - and involves between three and six testers.

You see the difference here? If I just want keep the scenario for myself and enjoy it on my own, I can do that right away. But if I want to share my scenario with you guys in the community, I´ll have to work twice as much on it – for months, literally.

And then, when I release the scenario, the feedback is mainly next to nothing. Here are some numbers from my own four hitherto released CMBN scenarios:

In total 1498 downloads that have recieved a total of 18 ratings or comments.

In other words: 1,2 percent of the users that have downloaded my scenarios have bothered to give me feedback (in reality even less, as the people rating often are he same as those that comment)

This is why I have begun to doubt whether releasing scenarios is really is worth the effort. Why bother when the response from the users is so meagre? I could have used all that time creating an entirely new scenario from scratch – for my own amusement.

I must stress that my reason for bringing this subject up isn´t about simply craving praise for my effort – even though praise is very nice to recieve, no doubt about that.

This is mainly about feedback. If I am to maintain any motivation for going on releasing scenarios, I need to know that they are being used, what the users think of them – and especially how they´ve played out. Nothing makes this designer happier than seeing one of his scenarios described in a thrilling AAR with lots of pics.

So what are the reasons for the lack of feedback - and how do we solve it?.

The reasons

I believe the main reason is structural. The scenario ”archives” do not encourage feedback from the users – more like the opposite.

I have nothing but praise for GreenAsJade´s great site, cmmods.com. Unfortunately, it doesn´t provide the ability to comment on the uploaded content. A pity, but I´d never complain: I´m just really happy that we have such a site for our mods and scenarios.

I believe that it is a major problem that The Repository is almost completely isolated from the Forum. There might be good reason for this, that I am not aware of. But in my view this is the main problem: You can rate and comment briefly on a scenario in the Repository, but if you want to post longer comments, screenshots, AAR´s, debate or ask a question, then you have to go a completely separate site, the Forum.

In my view, this greatly discourages feedback to the designers. These activities should in my opnion be integrated.

What to do?

If we are to establish effective user feedback, I believe we have to make the link between the scenario and a review/rating forum as direct as possible. The longer the distance between scenario and forum, the fewer comments.

One of the solutions I can think of, is to incorporate the Repository interface into the scenario thread in the forum. In this way you´ll be able to find and download the scenario in the same place where the scenario is discussed.

I´ve tried to construct a visualization here:

I also think the Repository generally needs improvement. In my experience the search function is close to useless in the Repository – and not very good in the forum either.

That means that even if you want to go back and rate or comment on a scenario, you might have a hard time actually finding the right file or thread.

Another solution could be this:

Make sure that you get the player´s attention exactly at the moment when he is probably most inclined to comment and/or review a scenario: When he has just finished it.

This is why I suggest giving the scenario designer the option to write a short message in the AAR screen. It could look something like this:

The optimal solution would of course be if this message could contain a link that would take the player directly to the discussion thread in the forum. But I bet BFC can think of a lot of problems with placing such a link.

Ian Leslie has suggested something along these lines too:

http://www.battlefront.com/community/showpost.php?p=1500231&postcount=40

This was my ”short” view on the feedback problem – and how we might solve it. I have no illusions that these suggestions will solve all problems and make the feedback rise dramatically in one blow. But at least it will remove some of the barriers.

I hope you survived reading through it – and that you´ll post your own views and suggestions here.

End of rant

Cheers

Umlaut

[Bimmer]

I sympathize, but I also believe - based on the pathetic amount of feedback on my own scenarios - that it's a lost cause no matter what you do to make it easier. You give people something for free, they have no motivation to do anything once they have it. Hell, I even tried to incentivize it: a while back I offered to upload a new scenario once I got three reviews on a newly-released one. Know what happened? I still, probably a year after the fact, haven't hit the magic three.

What it comes down to is this: I build scenarios because I like to. I build them the way I want, and if I feel like it, on occasion I release them. When I get feedback (good, bad or indifferent - I'm not just trolling for praise), I'm more inclined to release my work. Not surprisingly, I'm sitting on a bunch of tested and ready-to-go scenarios right now.

For those who complain about a lack of new scenarios, well, consider that - given the numbers of complainers versus the number of reviewers - you're probably not exactly helping the situation.

[Mr.X]

Hey Umlaut,

the whole thing seems to be like talking to a wall - you will hardly get any reaction from the community. I have made three campaigns (for CMBN, CMSF and CMFI), so I am able to know the effort, one has to put in such work - and 50% of the few comments, I got, have been negative At the moment, my campaigns all together have been downloaded > 1600 times - result: less than 16 comments - less than 1% of the users...so, following the theory, I have read in the other thread, they must be bad

That´s it. I will go on making campaigns and do not care about their statistics and one day in the future, I may decide to buy the BF Company and to hire a professional team of Scenario/campaign designers

Regards

Frank

[schwerpunktgrenadier]

I played Umlaut's Seizing Ciembienne about a month ago and thoroughly enjoyed it. One thing which stopped me from posting an indepth AAR or too many screenshots was that I didn't want to spoil it for anyone else. Maybe that stops others commenting on scenarios. But, when I see the amount of work that when into that beautiful map I do feel guilty for not posted a more thorough feedback. Perhaps designers could provide a quick questionnaire that could be rapidly filled in with every scenario? Or maybe we could get some of the experts at AARs on this site to give the rest of us a tutorial in producing structured responses textual and visual to aid our description of our experience playing a scenario. The figures clearly show that while most don't provide feedback they seem to be playing the scenarios at least.

[George MC]

Whilst feedback is really useful and I am fully appreciative of anyone (and I've been extremely lucky, and grateful to, have players who really have gone the extra mile to assist me in creating scenarios by providing top notch feedback) who takes the time to comment on a scenario I've made (for good, bad or indifferent) I'm simply happy to design stuff I enjoy playing. If others appreciate it that's excellent but TBH seeking feedback from players is not a motivator for me. I just enjoy creating the things!

[Dr.Fusselpulli]

Because there is a crucial difference between making a scenario that is playable for the designer himself and making one that is playable for everyone in the community, ie one that is ready for release.

In my experience the time need to make a scenario ready for release equals the time used on creating the scenario itself.

Making a scenario ready for release means:

- writing a briefing that is clear and hopefully entertaing to the players. Two briefings, if it is a H2H scenario.

- making a correct list of the forces involved

- making a strategic map BMP

- making a tactical map BMP

- making a preview BMP

- making a operational map BMP

- writing designer´s notes

This is so totally true. I too like to create new missions, but I never had uploaded one because of the named reasons.

Once I tried, and make the nice fictional scenario around Durrelôn, but as you said, made it community ready is a huge amount of work.

My english is terrible, so make a nice briefing is not an easy task to me. Also the testing of different IA plans brings me into a lot of trouble. For myself I have played my scenario together with a friend, where I can tell him directly whats the matter without a need of a nice briefing, but made it communty ready is not easy.

Also, I don´t have the 2.0 upgrade, so I am not sure if someone still would be interested in my scenario, so I never finished it to the community.

But I want to make some scenarios to the eastern front game, and I want to find some new partners who wants to play some PBEM-games. It sounds like a lot of fun. Maybe im not sure to make a scenario in the next time, but a kind of "mastermap". There I don´t need a briefing or a AI-Plan. Get more feedback is really important in my opinion.

[Frederico]

I am one of those who is reluctant to comment for fear of saying the wrong thing. I left comments on one scenario and was accused of a spoiler (which I did not believe was accurate). I think you know by the number of downloads that people are making use of your hard work, but I agree that more feedback should be provided. Honestly, the game is only as good as the new scenarios created.

[BletchleyGeek]

I do agree with Umlaut analysis and assessment. The question is whether this is something the community needs to take into its hands or rather requires action on behalf Battlefront.

Leaving aside that, your observation regarding 'what is the best moment to ask for a comment' is spot on - the best moment is right after the game is completed. Planting a link in front of the player's eye is one way, but of limited effectiveness I think. The link can be ignored.

But what about ladder playing? Right now I'm heavily engaged with theblitz.org "Scenario of the Month" event - playing, not organising - and I can't help but observe that when one needs to report the outcome of the scenario, a "comments" and "ratings" box included in the form to be filled in, might be much, much more effective as a device to gather feedback.

Of course, that only works for H2H - as we don't have a concept of a 'leaderboard' for single player. Maybe that's what you need, in order to "seduce" people playing the scenarios to speak up their mind.

[Davek555]

I know the old CM1 games had a dedicated site for scenario designers...I think it was called the "Scenario Depot"... where scenario designers could test and get feedback. Maybe something like that would encourage people to design more scenarios for CM2?

[transporter]

Like all good works of art, which I consider the construction of these scenarios to be, a good feedback indicator is the number of downloads. People love music, movies, art, etc. but very few will write the creator. Feedback usually contains critiques because human nature usually likes to point out areas of improvement. I have downloaded several scenarios and have enjoyed every one of them. Part of the decision to download is the number of times it has been previously downloaded. I guess I could have said something like how much I liked it, but never having made a scenario I figured I was not qualified to critique. Sometimes no news is good news. I am 66 years young and have played war games for before some of you were mere twinkles in your daddy's eye. So all you folks out there who spend countless hours making new battles for us to play keep heart. There are many silent ones like me who give you our heartfelt thanks!!

[c3k]

I read your thread and wasn't going to give any feedback, but I thought the irony would be lost on everyone but myself.

I agree with a lot of your points:

- The repository search function is horrendous.

- The download process is a bit of a pain.

- The work put in by many skilled contributors goes unrecognized.

- Feedback is more work.

Here's one part of what you wrote:

Another solution could be this:

Make sure that you get the player´s attention exactly at the moment when he is probably most inclined to comment and/or review a scenario: When he has just finished it.

This is why I suggest giving the scenario designer the option to write a short message in the AAR screen. It could look something like this:

The optimal solution would of course be if this message could contain a link that would take the player directly to the discussion thread in the forum. But I bet BFC can think of a lot of problems with placing such a link.

Ian Leslie has suggested something along these lines too:

http://www.battlefront.com/community/showpost.php?p=1500231&postcount=40

This was my ”short” view on the feedback problem – and how we might solve it. I have no illusions that these suggestions will solve all problems and make the feedback rise dramatically in one blow. But at least it will remove some of the barriers.

I hope you survived reading through it – and that you´ll post your own views and suggestions here.

End of rant

Cheers

Umlaut

Creating copy and paste link into the AAR screen which brings the player straight into the thread would be wonderful. (A hotlink would be better...)

I know that I have tried to post feedback when I can, but frequently entire SEASONS go by betwixt downloading something and playing it. The search function, time, laziness, etc, dampen any desire towards posting a response.

Your suggestion, quoted above, neatly bypasses that.

Perhaps allow an "AAR Designers Comment" section?

Thanks,

Ken

[Badger73]

I know the old CM1 games had a dedicated site for scenario designers...I think it was called the "Scenario Depot"... where scenario designers could test and get feedback. Maybe something like that would encourage people to design more scenarios for CM2?

This thread in general is timely for me. I am in the process of creating my first scenario intend to develop a several. I would welcome a "Scenario Depot" and suggest that it will and should include spoilers. Such a thread should include a caveat saying so.

The feedback I really need is twofold;

1) Playtesting a "beta" version so I can make it better for general release.

2) Players telling me both what they liked and what they disliked; particularly why.

I don't expect praise but when I screw the pooch, I'd appreciate knowing why and where things went south.

[Peregrine]

I suspect that the improvements in this engine for designers will increase the quality and number of scenarios.

[Callidus87]

Well, I can understand the OPs pain.

Well there are two things that jump into my mind.

First I dislike the repository (I have know clue how that translates into my native language)

The searchfunction is terrible. I never find what I want in that thing be it mods campaigns or scenarios. finding stuff again after a week is similiar difficult.

Also Comment sections aren't my thing. It is like writing on a wall, there is no interaction and you can't know if anyone reads it and/or understands what you mean.

That brings me to my second point.

You need a Board for stuff like this eithere a webpage with scearios and a Forum for them or an Forum board here where designers can introduce theire campaignes and scenarios and get feedback perhabs with AARs so that you have more eyecandy for that board.

[Ithikial_AU]

Agree the repository could be more user friendly both for mods and custom made scenarios. I'm also hopeful that the introduction of triggers encourages more people to take the plunge with creating just H2H maps but also fully fledged scenarios with AI plans but I wouldn't put money on it.

The move to CMx2 increased scenario editor time immensely from what my hazy memory remembers from CMx1 days. Anyone who's attempted an urban map before you'll understand. Is the extra effort worth it, absolutely, but agree it would be great to get better coverage of user generated content either on this site or some popular community 3rd party site, that's the best way to encourage people to take the plunge IMO.

[Erwin]

One issue may also be that we have seen designers react quite viciously to honest feedback which doesn't meet with the designer's desire for applause. Who knows how many have decided to say nothing.

[Jack Ration]

With the v3.0 engine almost on our doorstep it is an interesting time to talk "Scenario Design." The introduction of "triggers" should be a big plus in the scenario designers toolbox. Looking forward to see how this will playout.

I am hoping that "triggers" will make games vs the AI more interesting. After playing H2H it is a bit of a downer playing the AI as it currently stands. Considering the number of scenarios and QB options I am surprised anyone gets time to play additional scenarios. Although I do understand that a good designer will attract a "fan base".

I wonder how many of those downloads actually turned into games played.

Agree with the theme of finding an easy method of providing feedback to authors.

Maybe an unofficial Battlefront "Scenario of the Month" Directory - be nice if player's game results could be reported electronically to some type of scoreboard. Something like that might encourage players to download and play scenarios, whether H2H or vs AI.

Hardly a new or revolutionary software concept.

[Fizou]

First off I think that the amount of downloads in itself shows how well liked a particular scenario is. I also believe that most people will only rate if they are really blown away or disappointed. If they are content they probably won’t rate/come back to leave feedback. That’s just human nature.

I do sympathize with you umlaut and wish people would give more feedback to our talented scenario and campaign designers.

If we take a quick look at your threads for Ciembienne you have attracted a lot of praise and feedback in them. So maybe the forum should be used in a greater extent. Showing of pictures of a beautiful map and giving some hints about what the battle is about will have more players exited to try the scenario and gives them a easier way to leave feedback (not to mention that you will get reminded of a scenario when you see posts about it on the forum).

Other than that I hope people will take this to hart and try their best to give more feedback. Then it’s up to the designer to do what the like with that feedback. Just because one leaves feedback doesn’t mean the designer has to agree with it...

[agusto]

Maybe a lack of feedback means that the scenario is of, well, just standard quality, that it doesnt stand out among the rest. I know from my own commenting behavior that i only give feedback at the BFC Repository or here at the forums if something really impressed me, positively or negatively.

Another reason for why i dont often give feedback is because i often download lots of scenarios in one big 'Repository-looting-orgy' and then only play them months after downloading them. If something then doesnt turn out to be truely exceptional work, i dont really feel like i want to go look the Repository page up and leave a comment.

I personally make scenarios only for myself (or mostly) and most import for me is the feedback of the people who volunteer for testing, and they often give me very helpful and accurate feedback. For all the others, if they like my scenarios, that is great, if they dont, well, life is cruel. Feedback from non-testers is welcome but not really necessary for me.

[permanent666]

What me stopped from making scenarios was RL. I cannot afford the time to make a proper scenario. Every time I made a scenario or starting to do so i felt guilty investing so much time for a game. BTW I do not have the time to play CM either. It takes to much time to get immersed and due to micromanaging it still feels more like work than a game. But I love the game and how it evolves.

To biggest problem for me is doing the briefing. The process should be more straight-forward IMO. I want to upload the pictures in the scenario editor and edit them there. There should be symbols and arrows to apply on the pictures. And it would be great to write the briefing text inside the scenario editor.

Also placing flavor objects and buildings in 3D view would be a big step forward as bigger scenarios always take a lot of time to load on my machine.

[undercovergeek]

This thread made me go back and view my 2 scenarios from literally years ago - to be honest i made them as a labour of love to put back into the community and never expected anything in return - between them theyve had 1000 downloads and i have 2 or 3 encouraging comments

for me the biggest obstacle is time, i love map making, not so hot at the AI plans though, the problem being a perfectionist and constantly revisiting the map to adjust a barrel or a slope angle - it gave me a lot of pleasure designing the scenarios, and the comments i didnt know were there have made me smile this morning

i hope to be making more in the near future

Edit - i found the best way to get feedback was to do a WIP thread on the map section of the forum, people offer advice, get involved and then anticipate the release and drop feedback in the thread - maybe make a more active scenario design thread and encourage people to post about what theyre designing

[agusto]

Also placing flavor objects and buildings in 3D view would be a big step forward as bigger scenarios always take a lot of time to load on my machine.

If you have lots of RAM you could create a ramdisk. Install an instance of CM there and then create an image of that ramdisk so you wont have to go through the previous steps each time you want to launch CM from your RAM. That would speed up the process of loading levels or starting the game because no data has to be loaded from you harddrive.

[Glubokii Boy]

Edit - i found the best way to get feedback was to do a WIP thread on the map section of the forum, people offer advice, get involved and then anticipate the release and drop feedback in the thread - maybe make a more active scenario design thread and encourage people to post about what theyre designing

This is what i did when designing a map recently and i got very good feedback and comments on that one...helped alot !

link here...

http://www.battlefront.com/community/showthread.php?t=113050

But i obviously have to agree that the feedback and comments in general is few and far between...personally...I'm as guilty as most people regarding being poor at giving feedback...I will try to do better

[A Canadian Cat]

I played Umlaut's Seizing Ciembienne about a month ago and thoroughly enjoyed it. One thing which stopped me from posting an indepth AAR or too many screenshots was that I didn't want to spoil it for anyone else.

Interesting feed back. Don't be afraid @schwerpunktgrenadier if you label the thread what it is - and AAR then people will know the thread spoils the scenario and act accordingly. I have, on many occasions, skipped reading a thread that was on a scenario I was either playing or planning to play. I'll read them later. Just go for it - doing AARs on the forum can be quite fun too.

I am one of those who is reluctant to comment for fear of saying the wrong thing. I left comments on one scenario and was accused of a spoiler (which I did not believe was accurate).

Humm that is not good. This makes me think that feedback on a forum thread is a better way to go - because there is a bit of separation and you can declare that the thread contains spoilers.

[A Canadian Cat]

I´m starting this thread in the hope of opening a debate about the lack of feedback to scenario designers.

Good. I think a discussion about how to encourage comments is a good thing. I think we should acknowledge, though, that even if things are easy and the community offers encouragement the number of people who comment will still be small. It is kind of how we work (and by we I mean human beings). As we can already see from the comments of some scenario designers the threshold of feedback required to be motivated to create more varies from zero to higher than scenario designers are getting now. I suspect that even if we and BFC manage to make things better there will still be those that don't get enough feed back. Just saying

The optimal solution would of course be if this message could contain a link that would take the player directly to the discussion thread in the forum. But I bet BFC can think of a lot of problems with placing such a link.

Ian Leslie has suggested something along these lines too:

http://www.battlefront.com/community/showpost.php?p=1500231&postcount=40

I think that changes to the game and to the repository by BFC are probably the only thing that will truly have an impact. I repeat my suggestion linked above here for the record (this is a much better thread for this discussion IMHO). In the other thread Steve from BFC asked for ideas and I responded thusly.

OK how about this then. We know the key is to offer an opportunity to give feed back easily and at the right time (as in at the end of the game). How about having the game have a button that launches an URI which connects to the scenario's location in the repository. Or to a URI that auto appends to a forum post. That way the game change would be much simpler - get a URI from the scenario and let the OS find an app to open (aka browser) to take the user to a place that already exists.

Of course that does mean changes to the repository. But you could skip all that too by allowing the scenario designer to specify the URI and then they could start a forum post just before releasing the scenario and include that URI in the shipping scenario and boom for the price of a new property in a scenario file an a button to launch the browser on the end game screen you have a way to collect well timed feed back.

To make things even easier also put a command in the in game menu screen to connect to the scenario URI (same mechanism) that way it is easy while playing the game to comment on something right when it happens.