Script to automatically set the elevation in the editor

[RockinHarry]

4 hours ago, Butschi said:

Interesting, I thought Overath didn't see much more of the war than the train station being bombed.

that was just from my memory, so I could err and it was elsewhere in your area. I´ll reread some in Willi Mues big Der grosse Kessel (the big cauldron) book and maybe post a note or two soon here. Falls all within my area of interests anyay.

edit: no special mentioning of Overath to be found in there, but it most likely fell into the sector of US 97th Trident ID advancing north between 13. - 16th april 45. Either vs remains of german 353 ID or 3rd Para.

https://www.lonesentry.com/gi_stories_booklets/97thinfantry/index.html

and the US left some eggs. https://www.presseportal.de/blaulicht/pm/127076/4815255

Edited November 16, 2022 by RockinHarry

[Artkin]

On 11/15/2022 at 5:02 PM, Butschi said:

Unfortunately OSM doesn't have elevation data. There are a few resources in this post:

There are a lot of other sources and I can't possibly write conversion scripts for all of them, I'm afraid. I plan to write something for ALOS World 3D - 30m (AW3D30), it's free sattelite 3D data with a resolution of 30m. Not awesome but available for most of the globe. Generally, though, you should go for digital terrain models from airborne LIDAR scans that have been corrected for terrain features like buildings, etc.

Thanks, the link you posted was in German and I don't think i was able to translate the page via google translate. I will look around for the .csv files

[Butschi]

52 minutes ago, Artkin said:

Thanks, the link you posted was in German and I don't think i was able to translate the page via google translate. I will look around for the .csv files

Ah, yes, of course. Can you tell for which area you need the data? Then I can show you how to get there.

[Artkin]

2 hours ago, Butschi said:

Ah, yes, of course. Can you tell for which area you need the data? Then I can show you how to get there.

I was just going to piss around since I don't have any projects currently.

Areas of interest:

Kletskaya

Mamayev kurgan

Donetsk airport

preferably I'd like it to be the same size as my Editor Overlays:

[Butschi]

5 hours ago, Artkin said:

I was just going to piss around since I don't have any projects currently.

Areas of interest:

Kletskaya

Mamayev kurgan

Donetsk airport

preferably I'd like it to be the same size as my Editor Overlays:

Ok, that does look like we should indeed take a look at the ALOS data I mentioned before. Since I wanted to do that anyway, I'll write something to convert that kind of data - maybe over the weekend.

What you need to do: provide the coordinates of the bounding box for which you want elevation data. In your case that would be the corners of your overlay images, so the data extracted matches your image.

[SchnelleMeyer]

How is it going with the project Butschi?

[Butschi]

13 hours ago, SchnelleMeyer said:

How is it going with the project Butschi?

Recently I didn't have all that much time to spend on this... I'm still mainly working on getting buildings right. Single buildings are one thing but of course you want to have larger buildings that are composed of smaller buildings. There is an example of an industrial/commercial zone below.

You'll have to do the roof and textures yourself but the algorithm does its best to deconstruct larger polygons into smaller rectangles the size of different modular buildings in CM.

Also there are still those pesky linear features... Look at the conflict between road and railway above...

Anyway, I'm thinking about releasing a beta version soon (tm) so that I can gather feedback. (There is no way I can test all the fringe cases myself...). My main concern is having a good tutorial or something...

EDIT: I also looked into the ALOS satellite data for @Artkin . Unfortunately that kind of data is really not very good - It is raw elevation, meaning tree tops, buildings etc. which is not what we want here.

Edited December 5, 2022 by Butschi

[SchnelleMeyer]

Thanks a lot for the update and your work on this. I can't even begin to imagine how difficult this must be to write in code. Even if the user have to make adjustments to houses and linear features like roads, the framework will be there. It wouldn't take too much time to adjust it after running the script.

You might not have to do everything yourself. Like the tutorial - maybe some short instructions in a readme file is enough for a Beta release, and then maybe someone will pick up the glove and make a full video tutorial after a while. 

[Aragorn2002]

Really would like to master this. 

 

[TheFriendlyFelon]

Bro, this is freaking awesome. 

@Artkin was kind enough to share this post with me in a message, and I'm very glad he did. But like him, I am also clueless when it comes to code, so it looks like I will need to watch the tutorial. I do have a basic understanding of Lidar data, as I've used it for work once or twice, but I can't remember how I even did that. 

Thanks for making this man. This is sick.

[Bartimeus]

I just discover this....it's a game changing things you've done ! It's amazing. thanks for your work

[Butschi]

20 hours ago, TheFriendlyFelon said:

Bro, this is freaking awesome. 

@Artkin was kind enough to share this post with me in a message, and I'm very glad he did. But like him, I am also clueless when it comes to code, so it looks like I will need to watch the tutorial. I do have a basic understanding of Lidar data, as I've used it for work once or twice, but I can't remember how I even did that. 

Thanks for making this man. This is sick.

 

11 hours ago, Bartimeus said:

I just discover this....it's a game changing things you've done ! It's amazing. thanks for your work

Thanks! Be sure to also look at @kohlenklau's "test dummy" thread for how to actually get this thing to work.

[JM Stuff]

26 minutes ago, Butschi said:

 

Thanks! Be sure to also look at @kohlenklau's "test dummy" thread for how to actually get this thing to work.

Nico Check pls on your pm when you have time !

JM

Edited December 22, 2022 by JM Stuff

[Artkin]

Is the tool able to place houses and walls yet? Openstreetmap has a pretty detailed donetsk city, and I'd love to do a 5.7x5.7km chunk of the city. Specifically I'd be interested in this square:

Bottom right: 48.037494833658826, 37.76906093578662
bottom left: 48.03734556092855, 37.692193543741595
top left: 48.088658737254, 37.692385834418204
Top right: 48.088759104402335, 37.769123077955285

 

Edited December 22, 2022 by Artkin

[JM Stuff]

On 12/10/2022 at 3:20 PM, Aragorn2002 said:

Really would like to master this. 

 

Understand you wish and your pain, check to Phil infos about it, is available for everybodys, pehaps it will not work the first time like me but when you have some problems, there are enough peoples here to help you happy try !

JM

[Bartimeus]

8 hours ago, Butschi said:

 

Thanks! Be sure to also look at @kohlenklau's "test dummy" thread for how to actually get this thing to work.

So actually the version is with the import of data from Openstreet map? Hope battlefront hire you to implent a full system into the editor

[JM Stuff]

8 minutes ago, Bartimeus said:

So actually the version is with the import of data from Openstreet map? Hope battlefront hire you to implent a full system into the editor

We hope and show so much by differents thread what BFC can eventual added,that ours socks at the fireplace make be filled with good things for Chrismas or a little later

JM

[Butschi]

30 minutes ago, Bartimeus said:

So actually the version is with the import of data from Openstreet map?

Technically it processes geojson files that contain OpenStreetMap tags. The idea being that directly importing OpenStreetMap (directly downloading from the internet) is really only good for present day titles. CMBS and maybe CMSF2. Instead you use e.g. JOSM to download the data and modify it to your needs. For example, I use aerial imagery from ca 1980 as overlay in JOSM and just remove elements that weren't there 1980 or add those that were. Its easy to do, you draw a polygon and give it the "landuse: forest" tag. That way you can even create your own map from scratch and export it as OSM data.

But currently it is very rough around the edges and seeing how people struggle with just the elevations version, this really has to mature a little more.

[Dawntaker]

@Butschi This is amazing, wish I had found this before deciding to make a 4km by 3km Coldwar map.

On the accessibility side for the users that aren't experienced in the pains of software engineering a simple UI could be added in to replace the command line inputs, something like Tkinter or PyQt look promising for this. Packaging the python project will also help as it can be packaged with the requirements so people don't need to run pip or anaconda to install the necessary packages. 

I assume you probs haven't taken a look at if the thing works on other resolutions yet. This could be pretty easy to fix, just add in some of the popular resolutions people are likely to use, figure out the values needed for them and then add in a option/parameter to select one of them. 

Finally, I've taken a look at your code. Please comment it, I would say I'm pretty good with python even though I use Java and C# mostly in my job and I only understand half of it.  

If you want me to help I'm more then welcome to collaborate adding in more features, could try and tackle the a UI implementation for instance. 

[Butschi]

1 hour ago, Dawntaker said:

@Butschi This is amazing, wish I had found this before deciding to make a 4km by 3km Coldwar map.

Thanks! First post for commenting on this tool? I do feel honored.

1 hour ago, Dawntaker said:

On the accessibility side for the users that aren't experienced in the pains of software engineering a simple UI could be added in to replace the command line inputs, something like Tkinter or PyQt look promising for this.

I've considered it but it is not no. 1 priority right now. Tkinter and PyQt look like total overkill to me and the respective documentations are enough to not look any further at them. PySimpleGUI would be more than sufficient for that.

1 hour ago, Dawntaker said:

Packaging the python project will also help as it can be packaged with the requirements so people don't need to run pip or anaconda to install the necessary packages. 

Um... so I'm no expert on packaging but how would that make things easier? I mean, yes I could make a PyPi package and upload it but then you do "pip install cmautoeditor" instead of unzipping the code and doing "pip install requirements.txt", right? You still need a working python environment, and you still have to do that on the command line.

Also, CMAutoEditor needs packages that can't even be installed via pip. It needs conda packages or have the user manually install additional stuff which is no fun, either, so it would have to be a conda package.

The only way I see is using pyinstaller and make standalone packages, with the added benefit that you have clickable executables like so:

https://www.dropbox.com/s/goth529mnvxwkoh/cmautoeditor-1.0.1beta1.zip?dl=0

I've already asked people to try it but so far... no feedback.

2 hours ago, Dawntaker said:

I assume you probs haven't taken a look at if the thing works on other resolutions yet. This could be pretty easy to fix, just add in some of the popular resolutions people are likely to use, figure out the values needed for them and then add in a option/parameter to select one of them.

The code is tested with 1920x1080 and 2560x1440. @JM Stuff also managed to get it to work on 1024x768. Since apparently there is no UI scaling, the only thing that needs to be done is set PAGE_N_SQUARES_X and PAGE_N_SQUARES_Y to an appropriate value. Which is a fancy way of saying: Count the number of squares in the editor that fit on the screen and change above values accordingly.

[JM Stuff]

31 minutes ago, Butschi said:

 @JM Stuff also managed to get it to work on 1024x768.

31 minutes ago, Butschi said:

Just added with one and two screens, same resolution.

Thanks @ButschiTo your help about it.

Cheers

[Dawntaker]

Yeah first post, only got the account recently and saw this somehow. I've been around on the discord server run by Dom for a while but hadn't bothered to be present in other areas until now. 

17 hours ago, Butschi said:

I've considered it but it is not no. 1 priority right now. Tkinter and PyQt look like total overkill to me and the respective documentations are enough to not look any further at them. PySimpleGUI would be more than sufficient for that.

Fair enough, PySimpleGUI looks like a wrapper for Tkinter to simplify the code which will work fine for simple GUI's but could get messy for more complex ones. Like I said if you would like help then I'm volunteering to give it, to work on lower priority things whilst more important things are done by you. Seems to be something crazy with open street maps as far as I can see. 

 

17 hours ago, Butschi said:

The only way I see is using pyinstaller and make standalone packages, with the added benefit that you have clickable executables like so:

This is what I was thinking, hadn't realised you had already done it. I can give it a test when I get home. 

 

17 hours ago, Butschi said:

The code is tested with 1920x1080 and 2560x1440. @JM Stuff also managed to get it to work on 1024x768. Since apparently there is no UI scaling, the only thing that needs to be done is set PAGE_N_SQUARES_X and PAGE_N_SQUARES_Y to an appropriate value. Which is a fancy way of saying: Count the number of squares in the editor that fit on the screen and change above values accordingly.

That's fortunate, not much work needs to be done then, it would still be good to set some values for PAGE_N_SQUARES for each of the resolutions that would then be selected from an option that the user inputs. If the user needs to go into the code itself to set it for their specific resolution it wouldn't be a great experience. 

[Butschi]

22 hours ago, Dawntaker said:

Like I said if you would like help then I'm volunteering

By all means, give it a go, then!

The cleanest and easiest way is to fork my repository and make a pull request once you have something to share. Understand this, though: This is a fun project in my spare time. I make no promisses about when and even if I look into things and I won't be told how my code has to look and what should have comments, etc. I have enough of that at work.

22 hours ago, Dawntaker said:

it would still be good to set some values for PAGE_N_SQUARES for each of the resolutions that would then be selected from an option that the user inputs. If the user needs to go into the code itself to set it for their specific resolution it wouldn't be a great experience. 

True. Well, I have moving this stuff into config files and have "profiles" for different cm families and screen resolutions on my list. Which is growing longer and longer...

[kohlenklau]

3 hours ago, Butschi said:

I won't be told how my code has to look and what should have comments

Das ist ein codenführerbefehl!

 

[Dawntaker]

On 12/31/2022 at 6:34 PM, Butschi said:

The only way I see is using pyinstaller and make standalone packages, with the added benefit that you have clickable executables like so:

https://www.dropbox.com/s/goth529mnvxwkoh/cmautoeditor-1.0.1beta1.zip?dl=0

I've already asked people to try it but so far... no feedback.

I've given this a try and it works, none of the python libraries were required to be installed through pip or anaconda. The only problem that I can see tripping people up is that if they double click the exe to run it, it immediately closes the python window as there was no input parameter (so it errors). So the only way to currently run it is by opening a cmd window and then running the exe from that with parameters.