Creating Maps with CMAutoEditor

[Butschi]

I decided to create a new topic for this because "tool to set elevation in the editor" is no longer really a good description. Finally (from my perspective ) I managed to release version 2.0.0 of CMAutoEditor.

https://github.com/DerButschi/CMAutoEditor

Woah, Butschi, wait a minute CMAutoWhat? What the hell are you talking about? CMAutoEditor is a tool that takes "external" data, like elevation, and automates clicking in the CM scenario editor. This way it kind of works around "missing features" in the editor. Wouldn't it be nice to import something like a heat map instead of clicking elevations in the editor? Of course I can't change the editor but by letting CMAutoEditor do the clicking it does something similar.

 

Version 2 is really a major update as it now includes importing data from OpenStreetMap. This way you can basically use CMAutoEditor to create a whole map for you.

Still, feature creep and being overly ambitious is certain to kill such a small project so I had to triage what makes it into this release and what (maybe, provided there is enough interest on your side and enough time on my side) gets postponed to a later update.

The cut affecting most of you is that for this release only CMCW is really supported. The reason is that there are differences in what is in the scenario editor for each CM title. That means not only are there buttons at different positions but there are also different buildings, roads, flavour objects etc. which need to be programmed as a "profile". I intend to modify CMAutoEditor in a way that the community can help with that but I am not yet sure how best to do it. So, for now CMCW only. CMBS should work for everything but flavour objects.

Another thing I had to cut is support for Windows version older than Windows 10. I know, there are quite a few of you around who still field Windows 7. However, python is only supported up to version 3.7 on Windows 7 and a few of the libraries CMAutoEditor depends on require python 3.10. In addition, I don't have a farm of PCs idling at home where I can test different Windows version so I can really only develop for Windows 10. Sorry for that but maybe this doubles as a friendly reminder that nowadays Windows 7 is a security liability and you should upgrade to Windows 10 or 11.

There is of course a long list of things I'd like to improve eventually and given that this is a rather complex piece of code the chances that it is free of bugs is close to zero. So, feel free to contact me here or on github if you find something!

Now, with all the caveats out of the way let's move on to the more pleasant stuff. What's new in version 2? Well, of course OpenStreetMap data. You can import forests, rivers, roads, streams, railways, farmland, grassland, bushes, buildings and even bus stops, wayside crosses and much more. I improved the tools to import DGM data so that now arbitrary rectangles can be extracted. The information from this can be loaded into the OSM data converter so that you don't have to manually enter bounding boxes. (The geotiff tool is unfortunately among the things that didn't make it for this release so you are left with the previous capabilites). CMAutoEditor itself is now also much faster. Please note, that it may be too fast for what your PC can handle. If you experience "holes", e.g. when setting the elevation, that means the CM Editor is not responsive enough. I will make the parameters that determines the rate of clicking selectable from the GUI in a later update. Last but certainly not least there is a GUI for all the tools, so no more going to that evil shell, typing commands - kudos to @Dawntaker for supporting me with GUI making!

One more thing. I won't repeat the mistake and not have a manual or tutorial video ready at release: Since my previous tutorial video is now totally deprecated I am doing a new one. This time, instead of just telling you what the different features do, I'll do a step-by-step tutorial in four parts where I will create an actual map. This way I will certainly miss a few features but you should get a better feeling of how to actually use the tools in reality. Part 1 is finished already and it covers "installing" (i.e. downloading the .exe files) CMAutoEditor, installing other necessary tools, selecting a location for the map in map.army, finding and downloading elevation data, downloading OpenStreetMap data and getting historical aerial imagery and working with it. So, basically the boring stuff, some of you might say, but possibly the most important part.

In the future I'd like to adapt CMAutoEditor to one of the WW2 titles, I think CMFB or CMRT would be best. Obviously present-day OpenStreetMap won't do for that. So, instead I'd like to make another tutorial showing how to use and old map or aerial image to make "OpenStreetMap data" from scratch and feed it into the CM scenario editor. Drawing polygons and tagging them is still - in my view - much more comfortable than clicking in the editor with some semi transparent overlay. So, if you have good material, go ahead and make a suggestion here.

I hope you enjoy this new version of CMAutoEditor and the tutorial video(s)!

[Vacillator]

2 minutes ago, Butschi said:

I'd like to adapt CMAutoEditor to one of the WW2 titles, I think CMFB or CMRT would be best.

Yes please, that would be great .

[37mm]

Amazing work!

Funnily enough I've been thinking all of today about a project where your first version Auto-editor would have come in really handy... then I find out you've massively expanded your work & given it a GUI!

I believe, that thanks to the simplicity of hexediting, by covering CMCW you've probably already covered CMFB, CMRT & CMBS (CM Pro too) as all of these games have basically interchangeable maps.

@Artkin would know better but, if my thinking is correct, for maximum "bang for your buck" a CMBN or CMFI auto-editor would seem to be the most profitable.

Edited March 19, 2023 by 37mm

[Artkin]

1 minute ago, 37mm said:

I believe, that thanks to the simplicity of hexediting, by covering CMCW you've probably already covered CMFB, CMRT & CMBS (CM Pro too) as I believe all of these games have basically interchangeable maps.

Yep that was my thought too. CMCW is a great title because it plays nice with other titles more often. CMBS on the other hand is not as convertable friendly. 

5 minutes ago, 37mm said:

@Artkin would know better but, if my thinking is correct, for maximum "bang for your buck" a CMBN or CMFI auto-editor would seem to be the most profitable.

I must be the one of the only people in the world that don't enjoy CMFI. I'm not sure if my collection is smaller for that reason but it's not very big. 

IMO SF2/CMBN -> CMRT -> CMBS/CMFI -> CMCW -> CMFB in terms of sheer user content made for it.

In terms of active players I would swap CMFI with CMCW

[kohlenklau]

Thanks Nico!

[RockinHarry]

Gorgeous, Butschi! Yet need for WW2 titles as well (CMN, CMFB and CMRT)!

Btw, that would be the right moment for BFC to contact you and discuss/work out an interface! Well... earlier would´ve been even better.

Edited March 19, 2023 by RockinHarry

[DesertFox]

Excellent work!

[Aragorn2002]

CMRT would be the better choice. FB is already covered. 

[samuraiman]

This is very exciting! Excellent work!

[WimO]

16 hours ago, Vacillator said:

Yes please, that would be great .

OMG! Yes! I would also love to see this usable by CMBN. I would probably come out of map-making retirement.  Late war battles in the Netherlands anyone?

Edited March 19, 2023 by WimO

[Artkin]

9 minutes ago, WimO said:

OMG! Yes! I would also love to see this usable by CMBN. I would probably come out of map-making retirement.  Late war battles in the Netherlands anyone?

Well it can be, using hexedit. 

[WimO]

1 hour ago, Artkin said:

Well it can be, using hexedit. 

I have no such skill.

[Artkin]

I was following the video last night and when I got to geoportal.nrw I was having trouble finding any files on the map despite following your video 1:1. When I would select an area nothing would come up. I tried a few places for about an hour including Fulda and nothing came up. Only once did I get a hit but I skipped over it hoping a previously checked section would wind up working... which it didn't.

Do you have as much trouble as I do @Butschi?

Damn after typing all that I went to geoportal to check the section that worked and then my graphics card driver exploded.

I just checked a 30x30km square around Goslar now and had no luck.

3 minutes ago, WimO said:

I have no such skill.

For this case, you make the map in CMCW using Butschi's tool, then you open it up in hexed.it (A website). You click on one box with two numbers, change the two numbers and then export (Download) the file. It's very simple stuff and will add immensely to your games

[Butschi]

33 minutes ago, Artkin said:

I was following the video last night and when I got to geoportal.nrw I was having trouble finding any files on the map despite following your video 1:1. When I would select an area nothing would come up. I tried a few places for about an hour including Fulda and nothing came up. Only once did I get a hit but I skipped over it hoping a previously checked section would wind up working... which it didn't.

Do you have as much trouble as I do @Butschi?

Oh, simple answer: the nrw in geoportal.nrw stands for Northrhine-Westphalia. So this portal is only for the federal state Northrhine-Westphalia - and because this sounds like a super smart idea, every federal state has its own geoportal that works in a different way. Goslar is in Lower Saxony and Fulda in Hessen. Sorry, I should have mentioned and/or emphasized it.

For Hessen you can get the data here: https://gds.hessen.de/INTERSHOP/web/WFS/HLBG-Geodaten-Site/de_DE/-/EUR/ViewDownloadcenter-Start

Unfortunately it is, again, in German language only.

Ok, since for CMCW the Fulda region is important I'll give you an example: Click "3D-Daten" in the middle and then "Digitales Geländemodell (DGM1)" on the left side. And then... you need to know in which "Landkreis" (similar to county or district) the location is you are looking for. For that you can, for example,  go to google maps and right click somewhere in the area you are interested in. Let's say you want to redo "Döllbach Heights". Find the village and click "What is here?" somewhere there. You will be told that this is Eichenzell with a link to Wikipedia. Wikipedia will tell you that Eichenzell is part of the district of Fulda. Head back to where you were supposed to select the "Landkreis" and click "Landkreis Fulda". You will get a list of municipalities among which is Eichenzell. Download the file by clicking "Datei herunterladen". Don't worry if your map is on the border of two "Landkreise". You can just download the data for both and put them into the same folder (it will take longer to process, though).

[WimO]

A. I don't have CMCW to begin with.

B. Too much work for my old brain.

[Butschi]

16 minutes ago, WimO said:

A. I don't have CMCW to begin with.

B. Too much work for my old brain.

In that case, just weight until someone... well, I guess, me... does the profile for CMBN. It's just not on top of my list of priorities right now... Or, well I don't know. As I said above, if there is good elevation data available and you happen to have a good topographic map, it is a candidate for the next tutorial (since nobody made a good suggestion so far).

Edited March 19, 2023 by Butschi

[samuraiman]

Making huge quality maps is really difficult and time consuming. Hopefully this tool along with hexed.it (thanks Artkin) make the process much easier.

[Butschi]

Tutorial Part 2: Elevation

 

[samuraiman]

Do you know how to get the geodata for the area south of Cottbus? Brandenburg state? Spree-Neiße district?

[Butschi]

8 hours ago, samuraiman said:

Do you know how to get the geodata for the area south of Cottbus? Brandenburg state? Spree-Neiße district?

https://geobroker.geobasis-bb.de/basiskarte.php?mode=startup&aProductId=518094b9-d294-4e99-aeb2-550592a00682

Just mark the area you are interested in on the map and click "Einzelkachel (Info + Download)"

[DesertFox]

On 3/19/2023 at 12:52 AM, Butschi said:

In the future I'd like to adapt CMAutoEditor to one of the WW2 titles, I think CMFB or CMRT would be best.

I would suggest CM:FB. Availability of data for the Netherlands, Belgium, Rhineland, NRW, RP, Alsace should be no problem. Furthermore export of CM:CW maps via Hexedit to CM:RT works flawless. CM:SF2 is the only real troublemaker with Hexedit. Oh and dare I mention it. There is great coverage in literature for Commonwealth troops 10/44 - 04/45 also. I hope the CM:FB addon will get released end of this year.

Edited March 20, 2023 by DesertFox

[Redwolf]

Very cool project. Thank you.

[samuraiman]

16 hours ago, Butschi said:

https://geobroker.geobasis-bb.de/basiskarte.php?mode=startup&aProductId=518094b9-d294-4e99-aeb2-550592a00682

Just mark the area you are interested in on the map and click "Einzelkachel (Info + Download)"

Nice! Do you have a recommendation for Historical Aerial Imagery for the area? WMS link that you entered for nrw?

[Butschi]

7 hours ago, samuraiman said:

Nice! Do you have a recommendation for Historical Aerial Imagery for the area? WMS link that you entered for nrw?

Same way: https://geobroker.geobasis-bb.de/basiskarte.php?mode=startup&aProductId=140fca3b-18ba-49e5-928b-dc27ae522583

Next time its your turn to google, though. And please share your findings here.

[Butschi]

While I'm at it: If you use any of the tools would you please give me a little feedback? I have no clue if you all just find this stuff interesting but don't use it or if it is actually useful to someone. So something, anything, would be highly appreciated, be it a screenshot or just a few sentences about your experiences. Thanks!