Script to automatically set the elevation in the editor

[Butschi]

I wrote a little python script to set the elevation in the editor according to a digital elevation model. Those come in various resolutions, for instance, in Germany data for the whole of Northrhine Westfalia is available for free with a 1mx1m granularity. I then use PyAutoGui in order to automate setting the correcting height and clicking/scrolling in the editor. The script currently sets the height information for every square in the editor, not just contours.

An example height map:

The resulting terrain in the editor (3328mx1920m):

Vista somewhere in there.

The script is not terribly fast, making this map took about 7h, but it beats doing all the clicking yourself. The limitation, btw., is the editor, the script could go faster. The advantage is that you can do much better micro-terrain, I think, the disadvantage is that you can't just use an old topographical map and draw the contour lines. It would be possible to do contour lines but you'd need to have those in a machine readable format.

Anyway, if people here are interested, I could polish the code a bit and make it available to everyone, e.g. via github. If there is additional interest, I could imagine doing other stuff that way, like, for instance, drawing roads from Open Street Map.

[AlexUK]

Wow, looks like this has the potential to be an amazing tool!

[Commanderski]

I think there would be quite a few people interested in this, me included.

[Erwin]

If this about creating more realistic-looking terrain then it would be a great improvement.

[JM Stuff]

3 hours ago, Butschi said:

I wrote a little python script to set the elevation in the editor according to a digital elevation model. Those come in various resolutions, for instance, in Germany data for the whole of Northrhine Westfalia is available for free with a 1mx1m granularity. I then use PyAutoGui in order to automate setting the correcting height and clicking/scrolling in the editor. The script currently sets the height information for every square in the editor, not just contours.

An example height map:

The resulting terrain in the editor (3328mx1920m):

Vista somewhere in there.

The script is not terribly fast, making this map took about 7h, but it beats doing all the clicking yourself. The limitation, btw., is the editor, the script could go faster. The advantage is that you can do much better micro-terrain, I think, the disadvantage is that you can't just use an old topographical map and draw the contour lines. It would be possible to do contour lines but you'd need to have those in a machine readable format.

Anyway, if people here are interested, I could polish the code a bit and make it available to everyone, e.g. via github. If there is additional interest, I could imagine doing other stuff that way, like, for instance, drawing roads from Open Street Map.

 

Your script seem to have a long life here, this is also something that I would like to learn but don t know how can I begin, more please...

JM 

[Butschi]

Alright, I uploaded the script:

https://github.com/DerButschi/CMAutoEditor

It is still somewhat of a beta version, meaning it works for me but I only tested it with a screen resolution of 1920x1080.

Well, try it out, have fun and if you find something to fix or have suggestions, feel free to put it here or on github.

I added an example height map with which you can test if the script works or not. The end result should look like this:

[Bulletpoint]

I remember once suggesting that BFC add a random map generator to the game, working exactly like this. Generate or import a height map, than translate the shades into elevation points. I was told that idea was more difficult than I could possibly comprehend, so I'm glad someone went ahead and did it.

It might cause performance problems if you set an elevation point for every square of the map though, so maybe you could change it to only use 1/4 or 1/32 of the samples and let the engine interpolate.

Automatically adding clumps of trees and vegetation should not be too difficult either, but realistic looking towns and road would probably be a headache.

[Butschi]

6 minutes ago, Bulletpoint said:

I remember once suggesting that BFC add a random map generator to the game, working exactly like this. Generate or import a height map, than translate the shades into elevation points. I was told that idea was more difficult than I could possibly comprehend, so I'm glad someone went ahead and did it.

It might cause performance problems if you set an elevation point for every square of the map though, so maybe you could change it to only use 1/4 or 1/32 of the samples and let the engine interpolate.

Automatically adding clumps of trees and vegetation should not be too difficult either, but realistic looking towns and road would probably be a headache.

Randomly generating content would be a bit beyond the scope of this. At least I think developing an algorithm that yields a realistic result is difficult. But importing stuff from open street map would be doable. Forests, for instance. I think its just polygons and those are easy to translate into a grid. Roads would work, too. Buildings... maybe, though with a bit more effort, I guess.

So far I had no performance issues, at least not in the editor. Are you sure, there isn't a value for each square, anyway? My largest test so far was the map in my first post.

[Butschi]

On 9/24/2022 at 1:03 AM, Erwin said:

If this about creating more realistic-looking terrain then it would be a great improvement.

Well, it's as realistic as the data you feed in.

[Butschi]

Here is the btt-File for the above map. (Final Blitzkrieg)

example_map.btt

[Bulletpoint]

11 minutes ago, Butschi said:

Randomly generating content would be a bit beyond the scope of this. At least I think developing an algorithm that yields a realistic result is difficult. But importing stuff from open street map would be doable. Forests, for instance. I think its just polygons and those are easy to translate into a grid. Roads would work, too. Buildings... maybe, though with a bit more effort, I guess.

So far I had no performance issues, at least not in the editor. Are you sure, there isn't a value for each square, anyway? My largest test so far was the map in my first post.

 

You can just make the data by an online generator

https://fralonra.github.io/zatlas/

 

As for performance issues, I've read the developers state that the more elevation control points you use, the worse performance. I don't know if it's true, but always just took it as being true so I tried to minimise the amount of control points in my maps.

[Butschi]

2 minutes ago, Bulletpoint said:

As for performance issues, I've read the developers state that the more elevation control points you use, the worse performance. I don't know if it's true, but always just took it as being true so I tried to minimise the amount of control points in my maps.

The script will just set whatever data you feed it. If you provide it with only every 4th point or so it will happily take that. I've tested it with an input file that contained a single elevation datum.

[Bulletpoint]

1 minute ago, Butschi said:

The script will just set whatever data you feed it. If you provide it with only every 4th point or so it will happily take that. I've tested it with an input file that contained a single elevation datum.

That's brilliant. And of course there is no need to randomly generate data if you've got a source for real-world elevation data. That will tend to look more realistic than generated stuff.

[Bulletpoint]

One extra thing: setting elevation for every single square will create stair-stepping effects. This is because the editor only allows you to input a whole integer value for the elevation (1,2,3 etc. metres), but the algorithm for interpolating the terrain between the points can produce fractional elevations between them.

It's not a huge deal, your map looks generally good, but I think the terrain will "flow" better if you use fewer points.

Edited September 25, 2022 by Bulletpoint

[kohlenklau]

The only Python script I know is Monty Python. Thank you for doing your "black magic voodoo" sir. For future scenarios on a historical location, if the elevation data can be fed into your program, then it saves folks so much time & hassle in the CM map editor. One caveat being that sometimes a scenario/map designer kinda takes poetic license to change things based on limits of CM map editor road options and the like. 

[Butschi]

6 minutes ago, Bulletpoint said:

One extra thing: setting elevation for every single square will create stair-stepping effects. This is because the editor only allows you to input a whole integer value for the elevation (1,2,3 etc. metres), but the algorithm for interpolating the terrain between the points can produce fractional elevations between them.

It's not a huge deal, your map looks generally good, but I think the terrain will "flow" better if you use fewer points.

Yes, I think that is visible sometimes. It's a tradeoff between having micro-terrain with maybe stair-stepping effects and a somewhat too smooth and washed out features. Anyway, as I said, you can feed the thing whatever you want. I fear I am a better programmer than a scenario designer, so I hope for the experts to do something useful with it.

[Butschi]

By the way, there is highly accurate digital elevation data for more than just North Rhine-Westphalia. Here you can find data of Hessen (Fulda anyone?) for free here:

https://gds.hessen.de/INTERSHOP/web/WFS/HLBG-Geodaten-Site/de_DE/-/EUR/ViewDownloadcenter-Start

Edited September 25, 2022 by Butschi

[Vacillator]

On 9/23/2022 at 9:17 PM, Butschi said:

I wrote a little python script to set the elevation in the editor according to a digital elevation model...

I can't pretend that I understand every step of your process  but what you describe sounds great.  The end result also looks great.

Is there any indication of the performance hit from more control points?  Just curious...

[Butschi]

10 minutes ago, Vacillator said:

I can't pretend that I understand every step of your process  but what you describe sounds great.  The end result also looks great.

Is there any indication of the performance hit from more control points?  Just curious...

Really, there is no rocket science involved. I read in data from a file and automate mouse clicking. The script does nothing you couldn't do yourself, it only does so faster and you can let it run overnight or so.

Re: performance hit: I uploaded a the btt-File for the winter map a few posts above. Look for yourself.

Here is the file for the map I showed in the initial post: (Red Thunder this time)

example_map.btt

[kohlenklau]

@Butschi

I am asking this question, like a non-chef asking a chef how he made something to eat.

I THOUGHT these btt files were encrypted and we couldn't read the elevation data or even find where it is?

With the chef analogy, I may be beyond explaining this to. I admit it. If you must, just disregard the below question and stay on task sir.

With your knowledge of the btt file , can we see where panzerfausts are 1 or 2 randomly in a squad and somehow overwrite them to always be zero? I am an "early war" guy and this is an issue to making early war scenarios that don't all of a sudden have panzerfausts. 

[Butschi]

1 minute ago, kohlenklau said:

@Butschi

I am asking this question, like a non-chef asking a chef how he made something to eat.

I THOUGHT these btt files were encrypted and we couldn't read the elevation data or even find where it is?

With the chef analogy, I may be beyond explaining this to. I admit it. If you must, just disregard the below question and stay on task sir.

With your knowledge of the btt file , can we see where panzerfausts are 1 or 2 randomly in a squad and somehow overwrite them to always be zero? I am an "early war" guy and this is an issue to making early war scenarios that don't all of a sudden have panzerfausts. 

As described above. Sadly, I can't read the btt-Files either. I just automated scenario editor clicking. The script does what a human scenario designer would do. It is also not very fast (way faster than what I could do and I'd go crazy if I had to - I tried...) but as per my initial post, the map there took 7 hours. So, no, sorry, what you describe is nothing I can do.

[Vacillator]

4 minutes ago, Butschi said:

no rocket science involved. The script does nothing you couldn't do yourself

Ahem, yes of course.  I can create a map or edit another person's work and have done so as a fun exercise.  You just have to read the manual .

8 minutes ago, Butschi said:

performance hit: I uploaded a the btt-File for the winter map a few posts above. Look for yourself.

Sorry I saw that and missed the significance.  Thanks .

[kohlenklau]

11 minutes ago, Butschi said:

So, no, sorry, what you describe is nothing I can do.

No way, keep the trophy please. oh, that is ok. don't hang your head down at all, you did awesome. please say you are young AND smart so I don't feel old AND dumb...

Once again, best wishes to you and a sincere thanks for your good efforts and presentation here in the forum and actual accomplishment. I might use it down in the Kasserine Pass area of Tunisia.

 

[Butschi]

16 minutes ago, kohlenklau said:

No way, keep the trophy please. oh, that is ok. don't hang your head down at all, you did awesome. please say you are young AND smart so I don't feel old AND dumb...

Once again, best wishes to you and a sincere thanks for your good efforts and presentation here in the forum and actual accomplishment. I might use it down in the Kasserine Pass area of Tunisia.

 

Thank you for your kind words. I guess at age 43 I'm kind of in the middle... My girlfirend says I'm smart but I'm sure she only does it to make me feel good.

Let me know if I can help you with the Kasserine Pass. Out of curiosity I did a quick search and found there is elevation data available from ALOS on a 30m mesh. Not super precise but not to be scoffed at, either.

[kohlenklau]

53 minutes ago, Butschi said:

Let me know if I can help you with the Kasserine Pass

In a month or so  I will send you a PM. The 80th anniversary is just around the corner...I want to try to do some CMFI scenarios..