Creating Maps with CMAutoEditor

[kohlenklau]

https://data.nextgis.com/en/region/UA/dem/

[Butschi]

@TheFriendlyFelon@kohlenklau

As I pointed out a few times, beware of the difference between a digital surface model (DSM) (red line below), a digital terrain model (DTM) (cyan line below) and the ill defined term digital elevation model (DEM). A DSM is elevation including buildings, vegetation (tree tops,...), etc. A DTM is bare earth. A DEM can be either or anything in between (reaching into forests but not down to ground level, for instance.

For CM we obviously want a DTM. Now, the link @kohlenklau provided is based on ALOS satellite data. Satellite data is afaik always DSM or DEM, ALOS (at least ALOS 3D) certainly is.

Alas, I've been looking for a DTM of Ukraine for quite some time now. Seems like it either doesn't exist or is kept non-public (for obvious reasons maybe?).

Since DTMs are usually computed from DEMs/DSMs, it may be possible to generate the data manually but my tests so far were not very great.

[kohlenklau]

I understand all that Herr obercodenfuhrer!

I propose that if it is a rural area then how bad can it be for using red versus cyan from your image? I figure it is good enough for a CM scenario. 

[Butschi]

54 minutes ago, kohlenklau said:

I understand all that Herr obercodenfuhrer!

I propose that if it is a rural area then how bad can it be for using red versus cyan from your image? I figure it is good enough for a CM scenario. 

Rühren!

That was meant more for the benefit of @TheFriendlyFelon, since that website actually wants money (for data that is available for free on the internet).

You are right, in a relatively flat rural setting, that may be good enough. You'll probably want to adjust some elevations by hand but that is doable. But then: go get the elevation data for free. The dataset is called ALOS World 3D - 30m.

[kohlenklau]

I am a Windows 7 slacker and cannot use your CM Auto Editor tool.

But if I was able to, it would be a good experiment to make a test map with red and a test map with cyan and do a comparison to how much it impacts the final product.

Thanks Nico!

EDIT: And fear not, I WILL make scenarios with the 2 maps you made me for France 1940.  

Edited September 2, 2023 by kohlenklau

[TheFriendlyFelon]

Copy that. I decided to do my first map of the area the old fashioned way and spent about 8 hours on it last night. If I could expedite this even slightly it would help a lot.
 
I understand the differences in the elevation model types and I have actually noticed locations in Google earth with wonky elevations because they used DEM models instead of the DTMs. Ie; Hostomel, Ukraine has a bunch of imaginary hills that start at the forest edges around the south end of the town in Google Earth. (I actually had to go back and correct my map of the area when I got access to better info and realized my mistake.)

All that being said, is your program usable if I choose to skip the elevation data and only want to use it for the OSM data? If so, can you briefly explain how I can do that? You did a great job explaining how to import elevation data, but honestly, that's the easiest part for me to do on my own thanks to my career field (civil construction/civil design/earthwork estimator). Is there a way to just skip the elevation data, import the rest of the stuff, and then do my own elevations afterward? 

Please forgive me if thats a stupid question, I just don't really understand this process much and get lost any time I try. I was able to follow your tutorial video in a very linear fashion, but even that was a stretch for me. 

 

[TheFriendlyFelon]

@ButschiI attempted to just load OSM data to populate a map and it's not working. It says Preprocessing OSM Data:100% and then doesn't do anything.

[Butschi]

2 hours ago, TheFriendlyFelon said:

@ButschiI attempted to just load OSM data to populate a map and it's not working. It says Preprocessing OSM Data:100% and then doesn't do anything.

I assume you mean when running osm2cm? There are several parts that can take quite since time (without explicitly telling you so). The larger the map the longer it takes.

Anyway, as I keep reminding people, my telepathy skills are woefully lacking, so could you please provide me with a little more detail?

About your other question (probably related): Yes, you can just do the OSM stuff without elevations. In that case, when running osm2cm, instead of reading a grid file, you can choose to enter the bounding box manually.

Caveat: There is no CMBS profile (yet), so actually running cmautoeditor on the processed osm data will lead to wierd results. I always meant to add that profile but right now I'm a bit preoccupied with RL stuff. I'm not certain which of the other profiles is closest to CMBS and works best.
Depending on your coding skills you could contribute the profile yourself (it's not actually coding, more adding kind of lookup tables). Otherwise I'll have to ask you to have a little patience.

One more thing: Do you play the steam version?

[TheFriendlyFelon]

3 hours ago, Butschi said:

I assume you mean when running osm2cm? There are several parts that can take quite since time (without explicitly telling you so). The larger the map the longer it takes.

Anyway, as I keep reminding people, my telepathy skills are woefully lacking, so could you please provide me with a little more detail?

About your other question (probably related): Yes, you can just do the OSM stuff without elevations. In that case, when running osm2cm, instead of reading a grid file, you can choose to enter the bounding box manually.

Caveat: There is no CMBS profile (yet), so actually running cmautoeditor on the processed osm data will lead to wierd results. I always meant to add that profile but right now I'm a bit preoccupied with RL stuff. I'm not certain which of the other profiles is closest to CMBS and works best.
Depending on your coding skills you could contribute the profile yourself (it's not actually coding, more adding kind of lookup tables). Otherwise I'll have to ask you to have a little patience.

One more thing: Do you play the steam version?

Yes, that's what I meant. My apologies for not being more specific. I will try with a much smaller map and see if I have better luck.

I am pretty clueless when it comes to computer coding, but I am willing to attempt to help. Just don't expect a whole lot. 

No, I do not play on steam.

[TheFriendlyFelon]

@Butschi I got the OSM to CM conversion to work and successfully made a small map of my neighborhood in the United States with the auto-editor. I will attempt another one using elevations and let you know how that goes. I won't lie, I was kind of just staring at my screen in awe while the autoeditor was working it's magic. Well done!

[kohlenklau]

"CMAutoEditor works while you sleep and dream of your completed map served with coffee in the morning!"

[Butschi]

33 minutes ago, kohlenklau said:

"CMAutoEditor works while you sleep and dream of your completed map served with coffee in the morning!"

Still working on the coffee part.

[Butschi]

2 hours ago, TheFriendlyFelon said:

@Butschi I got the OSM to CM conversion to work and successfully made a small map of my neighborhood in the United States with the auto-editor. I will attempt another one using elevations and let you know how that goes. I won't lie, I was kind of just staring at my screen in awe while the autoeditor was working it's magic. Well done!

Very glad to read this. Meanwhile I took a look at what needs to be done for the CMBS profile. Seems like this is relatively similar to others so maybe it's not that much to do and I can do it myself.

[TheFriendlyFelon]

18 hours ago, Butschi said:

Very glad to read this. Meanwhile I took a look at what needs to be done for the CMBS profile. Seems like this is relatively similar to others so maybe it's not that much to do and I can do it myself.

So far everything I've asked it to do in Black Sea has worked.  It froze once and then glitched and kept repeating the same step another time, but both times it happened were on a massive map with a lot of data I was using to test it's limitations, so it was understandable. You did a great job making this thing. 

If we can find a way to get usable elevation data for Ukraine, that would be amazing, but honestly, so far it hasn't been too bad entering those on my own.   

[TheFriendlyFelon]

Ok, I'm trying to find elevation data for Ukraine and I'm realizing I have no idea what I'm doing. I found gray scale height maps of the area I'm trying to map and tried to follow along with your other video using QGIS but I keep getting an error code and I don't understand what is going wrong because I don't really understand what I am even trying to do. What kind of files am I looking for to start with? What am I converting them into? I'm trying to understand here but I am so confused.  

[nathangun]

Is this workable in CMBN?

[Butschi]

On 11/3/2023 at 12:28 PM, nathangun said:

Is this workable in CMBN?

Not yet fully, meaning the elevations stuff works (and France has good data) but there is no CMBN profile for processing OSM data, yet. Next release will contain a CMBS profile, afterwards I can take a look into CMBN.

[Artkin]

On 11/3/2023 at 7:28 AM, nathangun said:

Is this workable in CMBN?

You can just make it in another game and port it over to cmbn unless it's heavily housed in which case you might run into buildings with missing floors.

[samuraiman]

Huge map in the works of Gross Osnig area..

[Butschi]

6 hours ago, samuraiman said:

Huge map in the works of Gross Osnig area..

Thanks for sharing!

[Dawntaker]

A 2.5km by 2.5km map of Bad Hersfeld for Cold War that the auto editor has produced. Thanks for the awesome tool @Butschi the open street map part is mind blowing! Plenty of clean up work now to do before its finished. 

[Vergeltungswaffe]

Those last 2 map pics look very much like a real place™.

[nathangun]

On 11/6/2023 at 4:42 AM, samuraiman said:

Huge map in the works of Gross Osnig area..

It's good to get a map in JOSM that has all the fields with designated crops.

 

[Butschi]

Ah, it is the time of Merry Map Making again.

[Butschi]

I just released CMAutoEditor 2.3.

https://github.com/DerButschi/CMAutoEditor/releases/tag/2.3

The main feature is a new profile, CMBS.

In addition there are a few QoL and performance updates.

• Default foliage and ground settings: This was actually ninja-added earlier but I slightly improved the feature and added made it possible to switch them off. If you either add entries with tags ["default", "ground"] or ["default", "foliage"] and set their priority to -999, you can use these settings to replace the default tile mixture you get when starting a new map. I personally don't always like the patches of yellow grass and like to have a random tree in there every now and then. Every other item on that position will overwrite the default settings.
• Brush sizes: For things like ground tiles and some others (where appropriate), CMAutoEditor now uses larger brush sizes if possible. This greatly speeds up e.g. large areas of farmland.
• Minimum pause now selectable in GUI. This parameter is the minimal pause between two actions (e.g. clicks) in CMAutoEditor. If you encounter CMAutoEditor leaving out tiles that should be there (most likely visible when setting elevations on Steam), increase this parameter.
• Improved post-processing. The post-processing is the final step before writing results of osm2cm to file. If there is conflicting content for a tile, it takes care of properly sorting it out w.r.t. priority settings. It sometimes produced weird results and generally didn't communicate much. While working better now it may also be slower, so if you experience an excessive increase in processing time, let me know.
• Fixed a few bugs and missing features for selecting bounding boxes when running osm2cm. This significantly improves using osm2cm standalone, i.e. without loading a grid file from dgm2cm.

Enjoy!