OpenXcom Forum

Modding => Help => Topic started by: robin on March 27, 2021, 02:48:12 pm

Title: change the world
Post by: robin on March 27, 2021, 02:48:12 pm

I have three goals:
- modify the landmass so it matches a 70m sea level rise (see picture attached).
- add an urban texture (replacing a forest texture I guess, since there is an overabundance of them and I don't have a forest terrain); it is going to be used in small polygons representing mega-cities.
- adding some minimal improvements to make it look less blocky.

I'm not sure about the proper way to proceed.

I started tinkering with World Edit (by volutar), editing the included world.dat... is it how it's done?
There is a ruleset file with polylines: does it mean I have to change one by one these values, to match the modified polygons of my world.dat?

I tired to search the forum and I read some topics about globe editing, but the discussion was in general too advanced, I need a more "for dummies" introduction about this.

Any modder did this and wants to share the general steps?

Thanks

Title: Re: change the world
Post by: Buscher on March 28, 2021, 12:36:12 am

Did you try the World Editor/Change the World on Falko's (https://falkooxc2.pythonanywhere.com/worldeditor) yet?

When using this drop list "I want to edit" on the left side you can go down to import/export. There you can export the code for a rul file so you don't have to deal with the world.dat. When you go to the dropbox again you can click on terrain. In the middile it will say "no edit". Click on that. Now when you click inside one of the polygons you get the polygon points which you can drag around with left-click.

You can select which terrain you want to "draw with" with the dropbox in the middle when edit mode is on. Once you chose a "color" (f. ex. "Terrain - 9") you can click inside the polygons to change the terrain. To get a better idea what the terrain indices mean you can check the "textures:" section inside openxcom/standard/xcom1/globe.rul. I think that the ids are the same. For the urban texture you probably only need to change the name of one of the ids inside the "textures:" section to URBAN (see terrains.rul for details).

Title: Re: change the world
Post by: robin on March 28, 2021, 01:20:31 am

Thanks. The problem with Falko's tool is that I haven't found a way to put the image as a background, like I can do with World Edit, which is quite useful to have when you edit the polygons.
I'm probably still going to use Falko's to edit the other parts of the globe, like cities.

Anyway, I had missed the "data" entry for the globe, so I couldn't understand the polygons vs world.dat thing.

Title: Re: change the world
Post by: Buscher on March 28, 2021, 01:22:36 pm

Now I spent my morning to figure out how to do that what you can see in the attachment. The code is running in your browser so it should be possible and I even found a (hacky) way.

It works in Vivaldi. So what you gotta do is to open the world editor. Hit F12 to get to the web developer tools. Go to the console.
Paste the following and press enter

      marsterra = new OpenLayers.Layer.Image('marsterra', 'https://openxcom.org/forum/index.php?action=dlattach;topic=9554.0;attach=53436;image', new OpenLayers.Bounds(-180, -88.759, 180, 88.759), new OpenLayers.Size(288, 144),
                {wrapDateLine: true, attribution: "<a href='http://quanto.deviantart.com/art/Terraformed-Mars-53595798'>Provided by Quanto</a>"});
   map.addLayer(marsterra);

On the right side of the map there is a white plus with a dark blue background. Click on that and select marsterra. And now you should be able to trace your map with the world builder.

Title: Re: change the world
Post by: Finnik on March 28, 2021, 02:10:12 pm

Man, this is so cool! I was looking for a way to load background, but I failed both for World Edit and Falko's tool. I will try it)

Title: Re: change the world
Post by: Solarius Scorch on March 28, 2021, 05:01:02 pm

I really recommend worlddat - it's easy and fun to use. Falko's tool works, but is... neither.

Title: Re: change the world
Post by: robin on March 28, 2021, 06:18:25 pm

Quote from: Finnik on March 28, 2021, 02:10:12 pm

I failed for World Edit [...]

That's strange. I had an issue with a png, but two other jpg images loaded fine.
Maybe your images had the wrong resolution? As far as I understand, the width should be twice the height.

Title: Re: change the world
Post by: Buscher on March 28, 2021, 06:34:01 pm

Quote from: robin on March 28, 2021, 06:18:25 pm

That's strange. I had an issue with a png, but two other jpg images loaded fine.
Maybe your images had the wrong resolution? As far as I understand, the width should be twice the height.

Is there a working link? The download on the ufopaedia (https://www.ufopaedia.org/index.php/User:Volutar#World.dat_editor) doesn't exist anymore.

Title: Re: change the world
Post by: robin on March 28, 2021, 09:42:42 pm

The link works for me. At the bottom of the page.

Title: Re: change the world
Post by: Buscher on March 28, 2021, 10:15:12 pm

Hmm, interesting. I got HTTPS Everywhere running for Firefox and it will lead to an error page if the volutar page is accessed via HTTPS. Accessing it with HTTP-only will result in the download.

Title: Re: change the world
Post by: Finnik on March 29, 2021, 12:47:04 am

I think Volutar closed his server that was distributing his tools. For MCDEdit it was impossible to load from his link and I added mine (for dropbox, I hope he would not mind).

Title: Re: change the world
Post by: R1dO on March 29, 2021, 02:27:46 pm

As Buscher already noted, the problem is that his server (volutar.eu5.org) has no ssl certificate, hence any request starting with `https://` will fail.

The downloads (and thus server) are not offline, i could easily download both worlddat and mcdedit from the links (http://volutar.eu5.org/worlddat.zip and http://volutar.eu5.org/_mcdedit117j.7z) on the ufopaedia pages.

If you are unable to download chances are your browser is (forcefully?) trying to connect over `https://` instead of `http://`.

Title: Re: change the world
Post by: Finnik on March 30, 2021, 02:15:42 am

Quote from: Buscher on March 28, 2021, 01:22:36 pm

Now I spent my morning to figure out how to do that what you can see in the attachment. The code is running in your browser so it should be possible and I even found a (hacky) way.

It works in Vivaldi. So what you gotta do is to open the world editor. Hit F12 to get to the web developer tools. Go to the console.
Paste the following and press enter

      marsterra = new OpenLayers.Layer.Image('marsterra', 'https://openxcom.org/forum/index.php?action=dlattach;topic=9554.0;attach=53436;image', new OpenLayers.Bounds(-180, -88.759, 180, 88.759), new OpenLayers.Size(288, 144),
                {wrapDateLine: true, attribution: "<a href='http://quanto.deviantart.com/art/Terraformed-Mars-53595798'>Provided by Quanto</a>"});
   map.addLayer(marsterra);

On the right side of the map there is a white plus with a dark blue background. Click on that and select marsterra. And now you should be able to trace your map with the world builder.

Is it possible to do that with Chrome or Firefox browsers?

Btw, can I make non-vanilla texture with Falko's tool?

@robin - how did you prepared the map? i am looking for the background for Falko's tool wit geographic features (mountains, forests etc)

Title: Re: change the world
Post by: Buscher on March 30, 2021, 07:53:47 pm

It works with Firefox (just had a small problem with copy and paste that's why I switched to Vivaldi) and it should work for Chrome. Vivaldi is a Chromium, so it's nearly identical for this purpose.

What's possible is that when you copy from the spoiler that you will end up with 3 lines instead of 2.

This is the first command

Code: [Select]

var myImage = new OpenLayers.Layer.Image('myImage', 'https://openxcom.org/forum/index.php?action=dlattach;topic=9554.0;attach=53436;image', new OpenLayers.Bounds(-180, -88.759, 180, 88.759), new OpenLayers.Size(288, 144),);
The link to the openxcom forum is the link to the image.

and this is the second

Code: [Select]

map.addLayer(myImage);

Note: I have changed the name from marsterra to myImage.

Title: Re: change the world
Post by: robin on March 30, 2021, 09:46:37 pm

Quote from: Finnik on March 30, 2021, 02:15:42 am

@robin - how did you prepared the map? i am looking for the background for Falko's tool wit geographic features (mountains, forests etc)

I'm not sure I understand the question?
For editing the landmass, I'm probably going to use volutar's word dat editor, because it seems more practical; this tool has a button to add a background image.
Although I still a haven't figured out how to add/replace a texture with a custom texture... I need to understand that before doing anything.

Title: Re: change the world
Post by: Finnik on March 30, 2021, 09:58:48 pm

Quote from: Buscher on March 30, 2021, 07:53:47 pm

It works with Firefox (just had a small problem with copy and paste that's why I switched to Vivaldi) and it should work for Chrome. Vivaldi is a Chromium, so it's nearly identical for this purpose.

What's possible is that when you copy from the spoiler that you will end up with 3 lines instead of 2.

This is the first command

Code: [Select]

var myImage = new OpenLayers.Layer.Image('myImage', 'https://openxcom.org/forum/index.php?action=dlattach;topic=9554.0;attach=53436;image', new OpenLayers.Bounds(-180, -88.759, 180, 88.759), new OpenLayers.Size(288, 144),);
The link to the openxcom forum is the link to the image.

and this is the second

Code: [Select]

map.addLayer(myImage);

Note: I have changed the name from marsterra to myImage.

Looks working, thanks! And how do you get those values, i imagine they meant to adjust the map. I am looking to add this one as background to edit biomes
https://u-stena.ru/upload/iblock/28b/28b861daa37da95f4740d1f0599ea8e2.jpg

Quote from: robin on March 30, 2021, 09:46:37 pm

I'm not sure I understand the question?
For editing the landmass, I'm probably going to use volutar's word dat editor, because it seems more practical; this tool has a button to add a background image.
Although I still a haven't figured out how to add/replace a texture with a custom texture... I need to understand that before doing anything.

Well, the question is that because all maps I am loading are placed wrong, but I guess I need to adjust values in script.
Yes, I also can see the problem with Falko's tool that I can't add textures to it. But on another hand, I can't add a background picture to Volutar's tool - it always throws an exception. And I need both =)

Title: Re: change the world
Post by: Buscher on March 30, 2021, 10:17:14 pm

Like in the attachment?

You don't have to mess with the parameters. You just have to make sure your picture has the correct map projection (https://en.wikipedia.org/wiki/Map_projection). Frankly I don't know what the numbers do either. But you can look up the used API and its parameters.

In your case it's quite simple as you only have to resize your picture to the size of robin's map (1280x714). That's exactly what I did to your map. Perhaps using this size you can also put it into Volutar's.

The arctic and antarctic are messed up because of the not correct map projection for this application.
No idea about how to add new terrain layers though.

Title: Re: change the world
Post by: Finnik on March 30, 2021, 10:27:25 pm

Omg, I manage to load it to Volutar's tool! I thought it's broken!
Tho the map needs the same correction now it looks like I am trapped, as i did a lot of work with Falko's tool, that export rulesets, and Volutar's  use only .dat file. Darn  :-X

(https://media.discordapp.net/attachments/701187824703242320/826264626051809300/unknown.png?width=1017&height=801)

There should be the way to convert from one to another, I believe, as the essential data there is the very same. Halp!  :'(

Title: Re: change the world
Post by: Finnik on March 30, 2021, 10:59:09 pm

Looks like that if you manage to edit texture on the polygon directly, Falko's tool can load and edit it, but you can't create new of those. My guess is that it can't load new textures. But on its forum page it shows it can. So I am confused. Also, guys, can you load some Earth backgrounds other than WMS? And is there a way to load, say, Google Maps with it?

Title: Re: change the world
Post by: robin on March 31, 2021, 11:27:37 pm

Yeah, I don't understand how textures work either.

A conversion tool would be nice.. I once made a python script to mess with RMP files... maybe I can mess with world.dat file too, to make a yaml out of it.
Nor sure if I can do it, I never fully understood the bytes thingy.

Title: Re: change the world
Post by: Finnik on April 01, 2021, 01:41:01 am

Quote from: robin on March 31, 2021, 11:27:37 pm

Yeah, I don't understand how textures work either.

A conversion tool would be nice.. I once made a python script to mess with RMP files... maybe I can mess with world.dat file too, to make a yaml out of it.
Nor sure if I can do it, I never fully understood the bytes thingy.

Well, can be an easier way - Volutar's tool can export to .xyz file. I really can't understand that choice, as IMO it is the worst format to represent GIS data. I worked a little with cartography once I was geologist, and there are a lot of nice formats, like shape files. XYZ file is text file with coordinates in 3d, but the shape is flat, z is always 0.
Anyway, as Volutar's too can export and import to it, I would say it is better than dealing with binary file.

Title: Re: change the world
Post by: robin on April 01, 2021, 10:41:31 pm

Quote from: Finnik on April 01, 2021, 01:41:01 am

Well, can be an easier way - Volutar's tool can export to .xyz file. I really can't understand that choice, as IMO it is the worst format to represent GIS data. I worked a little with cartography once I was geologist, and there are a lot of nice formats, like shape files. XYZ file is text file with coordinates in 3d, but the shape is flat, z is always 0.
Anyway, as Volutar's too can export and import to it, I would say it is better than dealing with binary file.

AFAIK the geoscape is actually flat, so it makes sense.

Title: Re: change the world
Post by: Yankes on April 01, 2021, 11:05:04 pm

Quote from: robin on April 01, 2021, 10:41:31 pm

AFAIK the geoscape is actually flat, so it makes sense.

No fully accurate, you define shape on flat surface but game project it on surface of globe that is not flat (but still 2d surface).
This have consequence that near poles projection break because on globe you have connected triangles but on 2d flat surface you have disconnected ones.
Simply upper edge is converted to single point.

I do not check this code recently but there is possibility that you could find "holes" in map when you are close to pole.

Title: Re: change the world
Post by: Finnik on April 01, 2021, 11:29:36 pm

Well, yes, very standard projection is used to represent a globe on a flat surface.
I assume converting xyz to rul and back is an easy task for python scripting, but I literally know nothing about python =(

Title: Re: change the world
Post by: robin on April 02, 2021, 12:44:31 am

I wrote a python scritp to extract the coordinates from world.dat and put them into a rul file.
I don't understand why, all the X coordinates are wrong while all the Ys are correct.

Example, First polygon -
my script:

Code: [Select]

[2799, -416, 2804, -411, 2828, -418, 2818, -425]world.dat:

Code: [Select]

0: 1279,-416
1: 1284,-411
2: 1308,-418
3: 1298,-425
Both coordinates should be short integers made up by 2 bytes, according to https://www.ufopaedia.org/index.php/WORLD.DAT

Title: Re: change the world
Post by: robin on April 02, 2021, 12:47:24 am

Quote from: Finnik on April 01, 2021, 11:29:36 pm

I assume converting xyz to rul and back is an easy task for python scripting, but I literally know nothing about python =(

I don't understand. I checked volutar's tool and I can only export to .obj format.

Title: Re: change the world
Post by: Finnik on April 02, 2021, 01:29:47 am

Quote from: robin on April 02, 2021, 12:47:24 am

I don't understand. I checked volutar's tool and I can only export to .obj format.

Right, my bad, .xyz is a format I was messing around to convert from. Well, .obj is pretty much the same. Its the text file, first part contains coordinates, the second, I assume, holds texture data (how to construct polygon and set texture ID) with links to vertices numbers.

Title: Re: change the world
Post by: Buscher on April 02, 2021, 10:01:57 am

Aren't the coordinates for the globe in Polar coordinate system (https://en.wikipedia.org/wiki/Polar_coordinate_system)? So essentially longitude, latitude and fixed radius? Longitude has a range of 360 with an accuracy of 1/8 (0-2879) and Latitude has a range of -90° to 90° with an accuracy of 1/8 (0-719 and 0x10000-0x2D0). No idea how it is projected on a 2D map though.

Quote from: robin on April 02, 2021, 12:44:31 am

I wrote a python scritp to extract the coordinates from world.dat and put them into a rul file.
I don't understand why, all the X coordinates are wrong while all the Ys are correct.

Example, First polygon -
my script:

Code: [Select]

[2799, -416, 2804, -411, 2828, -418, 2818, -425]world.dat:

Code: [Select]

0: 1279,-416
1: 1284,-411
2: 1308,-418
3: 1298,-425
Both coordinates should be short integers made up by 2 bytes, according to https://www.ufopaedia.org/index.php/WORLD.DAT

I am confused by your X numbers of the world.dat. The first polygon of UFO/GEODATA/WORLD.DAT for me is
2796, -418
2804, -411
2828, -418
2806, -426

UFO/GEODATA $ cat WORLD.DAT | hexdump -C | head
00000000  ec 0a 5e fe f4 0a 65 fe  0c 0b 5e fe f6 0a 56 fe  |..^...e...^...V.|
00000010  01 00 00 00 0c 0b 5e fe  0e 0b 51 fe fa 0a 4e fe  |......^...Q...N.|

(Little Endian)

ec 0a           2796                349,5°
5e fe           65118 (-418)    - 52,25°
f4 0a            2804                350,5°
65 fe           -411                  - 51,375°
0c 0b           2828                 353,5°
5e fe           -418                  - 52,25°
f6 0a            2806                 350,75°
56 fe           -426                  - 53,25°
01 00 00 00

Title: Re: change the world
Post by: robin on April 02, 2021, 10:09:43 am

Quote from: Buscher on April 02, 2021, 10:01:57 am

Aren't the coordinates for the globe in Polar coordinate system (https://en.wikipedia.org/wiki/Polar_coordinate_system)? So essentially longitude, latitude and fixed radius? Longitude has a range of 360 with an accuracy of 1/8 (0-2879) and Latitude has a range of -90° to 90° with an accuracy of 1/8 (0-719 and 0x10000-0x2D0). No idea how it is projected on a 2D map though.


I am confused by your X numbers of the world.dat. The first polygon of UFO/GEODATA/WORLD.DAT for me is
2796, -418
2804, -411
2828, -418
2806, -426

UFO/GEODATA $ cat WORLD.DAT | hexdump -C | head
00000000  ec 0a 5e fe f4 0a 65 fe  0c 0b 5e fe f6 0a 56 fe  |..^...e...^...V.|
00000010  01 00 00 00 0c 0b 5e fe  0e 0b 51 fe fa 0a 4e fe  |......^...Q...N.|

(Little Endian)

ec 0a           2796                349,5°
5e fe           65118 (-418)    - 52,25°
f4 0a            2804                350,5°
65 fe           -411                  - 51,375°
0c 0b           2828                 353,5°
5e fe           -418                  - 52,25°
f6 0a            2806                 350,75°
56 fe           -426                  - 53,25°
01 00 00 00

I don't know why my worlddat editor shows incorrect X values (see screenshot).
Anyway I managed to complete the script, it produces a working "polygons" list to copy-paste into globe.rul. I just need to test that it imports correctly on Falko's tool.

Title: Re: change the world
Post by: Buscher on April 02, 2021, 10:57:55 am

For some reason there seems to be an offset of 1520.

Title: Re: change the world
Post by: Finnik on April 02, 2021, 08:24:12 pm

Quote from: robin on April 02, 2021, 10:09:43 am

I don't know why my worlddat editor shows incorrect X values (see screenshot).
Anyway I managed to complete the script, it produces a working "polygons" list to copy-paste into globe.rul. I just need to test that it imports correctly on Falko's tool.

That is supercool, I would really welcome this addition!

Title: Re: change the world
Post by: robin on April 05, 2021, 01:14:19 pm

The problem is that Falko's tool was written long ago and the ruleset evidently changed since then (for example: cities are defined in a completely different way). So, to complete this properly, I'd need to make a conversion. I didn't take this into account when I started, I hoped it were more straightforward. Therefore I don't know when/if I'll be able to do this.