Author Topic: Improving World.DAT discussion  (Read 15178 times)

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #15 on: August 30, 2018, 09:57:35 pm »
Found the three bad polys, thanks for the spot. Only the three, correct? If it's what you want, its yours

Thank you very much!

However, I've found more bad polys once I installed it, check attached image
« Last Edit: August 30, 2018, 10:07:43 pm by Hobbes »

Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #16 on: August 30, 2018, 11:12:16 pm »
Found those three as well, then actually booted up current progress to confirm BEFOREHAND that there weren't any more.
I THINK  ;) this version is error free, attached below.

Couple days of good pushing and the western hemisphere is roughed out, onto Europe and Africa!

« Last Edit: August 30, 2018, 11:13:48 pm by SophiaThe3rd »

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #17 on: August 31, 2018, 12:58:04 am »
Found those three as well, then actually booted up current progress to confirm BEFOREHAND that there weren't any more.
I THINK  ;) this version is error free, attached below.

Couple days of good pushing and the western hemisphere is roughed out, onto Europe and Africa!

It is error free, just checked ;)

This is excellent, still a lot of work involved but the old Geoscape is finally gonna get replaced

Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #18 on: March 06, 2019, 12:07:30 am »
Work had slowed some these last few month while i sorted things out, but nevertheless I have hit what I consider to be another milestone. The entirety of the globe is rendered with coastlines, lakes, flatlands, hilly/mountainous areas, steep mountain, ice flows and polar caps. A LARGE number of fixes and adjustments have been made to combat the significant amount of overlaping polygons in previous versions. Not completely eliminated, but much more under control. Next is deciding exactly how to apply the textures to the terrains in a fashion that makes sense, is easy to modify, and produces a pleasing look to the globe. Also still to do is cleaning up some of the country polylines where coastlines have been altered. All that said, it occurs to me that this current build might be useful to someone in the future as a sort of "proto-earth" or post apocalyptic base build ready to scorch. ;)

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: Improving World.DAT discussion
« Reply #19 on: March 06, 2019, 12:21:43 am »
This is a great accomplishment and I salute you.

My only hope is that you saved most of the intermediate data and that you're going to post it too.

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #20 on: March 06, 2019, 01:22:06 am »
Just had a look at it using Tech-Comm and it looks amazing. Good work!

Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #21 on: March 06, 2019, 02:04:00 am »
intermediate data? something specific? there isnt much in the way of incremental versions i'm afraid.. so far anyway. Most of whats done thus far is a massive upshot of polys and their placements. I didnt think that a version with the Americas done but Africa and EurAsia not,per se, would be terribly useful to anyone.

Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #22 on: March 06, 2019, 02:05:33 am »
screen cap of Asia

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: Improving World.DAT discussion
« Reply #23 on: March 06, 2019, 02:17:49 am »
intermediate data? something specific? there isnt much in the way of incremental versions i'm afraid.. so far anyway. Most of whats done thus far is a massive upshot of polys and their placements. I didnt think that a version with the Americas done but Africa and EurAsia not,per se, would be terribly useful to anyone.

Please share how did you accomplish this.

Was it all done by hand-drawing polygons?

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #24 on: March 06, 2019, 04:28:18 am »
North America using the latest version


Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #25 on: March 06, 2019, 06:15:22 am »
Fear not, I do intend to sit down and write out a documentation of the entire process and revelations ive had along the way. In the mean time, huge thank you to all the devs and community that make things like this possible in the first place. If only system shock saw this kind of love...

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #26 on: March 08, 2019, 05:18:40 am »
Just had a look using world.dat editor and the level of detail is impressive. About assigning textures to the polygons I think you're in the right track by starting to differentiate between low/high altitude areas, with different Geoscape sprites to each.

Offline Hobbes

  • Commander
  • *****
  • Posts: 2102
  • Infiltration subroutine in progress
    • View Profile
Re: Improving World.DAT discussion
« Reply #27 on: March 09, 2019, 05:03:18 pm »
OK, after some thinking here are a couple of pointers that might help with the development:

The polar regions are lacking the detailed coastline polygons, which I think should be added because the visual difference is striking. I'm wondering if you didn't add them because they're ice and thus constantly change over the years.

The next step I'd take after polygon design would be to start choosing visual textures for textures.png and try them out to see how they look, and how they look together. The main issue here is that the Geoscape palette is quite limited, plus the day/night cycle uses different gradation coloring, so there aren't many choices available there regarding colors.

The lack of color variety will have effects on the realism, if you check an orbital picture of Earth the colors will be too bright and the polygons will stand out, but this can be mitigated by choosing different visual patterns for each texture. And the lack of realism is also due to other factors that are impossible to control, like seasonal changes (winter in the northern hemisphere and summer in the southern one), plus elevation differences are not always visible from space. But here I think that realism needs to take a step back to aesthetics because if the globe doesn't look good visually that is going to be an eye sore for the player.

Once the visual patterns are chosen and assigned to polygons.png, then it will be a matter of assigning those textures to the individual polygons, and you're probably cringing right now because there are thousands of them. One way to reduce the workload is to assign to texture #0 the most used visual pattern, and then manually edit the other textures.

But here you'll also have to take a design decision: either only use the original textures, which will allow your mod to be used easily by anyone by simply replacing the world.dat file; or add new textures, either from the Terrain Pack or whichever you like, giving you more flexibility but making the modders have to adapt to your system.

This decision also applies to the terrains being used: you either stick to the vanilla ones, or you add additional ones from the Terrain Pack or elsewhere to represent the new biomes, etc. However, since the terrains are assigned through the rulesets, modders can easily choose which ones they want to assign to each texture. The important thing to remember here is that the two systems (visual Geoscape textures and terrains used) are used together, but are independent from one another regarding design: one is edited through world.dat, the other through the ruleset. So if world.dat is finished, then someone else can take it and adapt it to their needs regarding terrain assignment.

There are a couple of issues that modders will also have to fix, and I'd advise leaving those fixes to them, because they involve changes in regions.rul: city placements and missionZone definitions (so that UFOs don't land on the ocean/lakes)

Finally, about the terrains used, with the Terrain Pack I've designed it like this:
* There are several biomes: Farm, Forest, Desert, Jungle, Mountain, Polar, Tundra, Taiga, Grassland, Steppe and Savanna
* Each biome can have different terrain versions assigned. This is done by reusing existing sets of mapblocks and giving them a different visual look by assiging different MCD sets
* The existing generic sets of mapblocks are: DESERT.MAP, FOREST.MAP, DESERTMOUNTAIN.MAP, FORESTMOUNTAIN.MAP and FORESTPOLAR.MAP. By using different MCD sets to them, I can create 5 subterrains to most biomes, each using different mapblocks.
* There are a number of unique sets of mapblocks for terrains where I couldn't do the previous trick: the temple versions of Jungle and Desert, the Jungle Mountain mapset, the different version of Farm, etc.
* These terrains are then assigned to each texture,  according to their distribution in real Earth, but also trying to keep their appearance balanced

Offline SophiaThe3rd

  • Sergeant
  • **
  • Posts: 23
  • For Love of the Frog
    • View Profile
Re: Improving World.DAT discussion
« Reply #28 on: March 25, 2019, 06:24:16 am »
Awesome post. Insightful and informative.

The polar regions are indeed unfinished. Currently they sit at a confirmed "stable" configuration while experimenting with my approach to polar ice flows. I have period accurate reference photos of polar ice min/max states for 1999. An issue is that at such high latitudes polys become less malleable in world editor. It's a pain, and on backburner until I hammer out the structure of my textures and terrains.

For textures.png I still haven't come down on a side of the fence for OG/TP4, but either way realism was always meant to be in the play and not on the visual. I also expect that either way the elevations so plainly visible now will be almost swept away visually by the end, but still there when it comes to crash sites.

By the time I am confidant in my framework, I think texture painting will be fairly smooth, and progress will be rewarding immediately.

I think the complexity of the interplay between the systems involved (textures.png, rulesets, world.dat, mission zones, etc..) limit the usefulness of using a world.dat replacement alone. Not advised but certainly not forbidden.

I have always meant for the final version to be properly boxed up, and able to be flipped like a light switch in the mod menu, unaffected by all other (non-geoscape) mods. To achieve this I don't see how I COULDN'T address the missionZones, polylines for borders, and city placements.

Your final blurb is forum gold. VERY useful explanation.

some observations:
the original structure is a mess when it comes to the terrains and textures. all four textures for forests share with jungles, but farms have four textures all to themselves. even polar and desert get two apiece. this jamming together is confusing at best and at worst should probably be undone, maybe by a tiny mod of its own merit.

while they share colors on the globe, forests can only occur in the north and jungles only in the south. this is just wrong and the system used to do it is of questionable usefulness at all.

Forest/Jungle texture 11 is totally unused, and is probably something that was cut for production times but more interesting is Forest/Jungle texture 6. This texture is only used twelve times in the original globe. I speculate (based on their placements) this might have been a different unfinished terrain that was cut entirely, perhaps swamps.

The MCDs for desert, forest and mountain all already have elevated terrain for battlescape use. Amending these onto farm and polar MCDs and adding some ground surface art for each gives all OG biomes both flat and hilly capabilities. A few extra mapBlocks will need to be assembled to accommodate the behavior change. Maybe a mapScript as well. For mountainous biomes I can just copy the other 5 terrains ground surface .pck art onto duplicated copies of mountain itself. Bring along a few trees, stumps, or grasses from each and BAM. Existing mapBlocks and mapScripts should transition perfectly. I suspect that much of this is already done in TP4.0, I haven't dug in that deeply yet.

When all is said and done, this mod should be easy to modify and easy to add to. It will be nearly impossible to rearrange or reorganize. Because the original structure is a mess and because I have future modders (myself and others) in mind, extra EXTRA care is being given to layout. Original programmers were much more tidy with organization on systems such as alienDeployments itemLists or the research tree. As such these various other systems were MUCH easier for me to understand and imagine improvements for. It will greatly reduce future headaches if this thing is easy to read.

Any all-new textures will require art that not only looks the part, but also wraps around on itself on all four sides. I don't know if I can do that. I believe there is ample art already available in TP4, and the texture decisions all boil down to the "look" of the globe. OG or TP4. Again, I still haven't decided. It MAY be possible to upgrade a finished OG version into a TP version(visually speaking) but only if I get the layout right the first time, as the visual elevation references i have in prep for painting will be gone after said painting.