OpenXcom Forum

Contributions => Programming => Topic started by: pmprog on July 06, 2010, 12:10:13 am

Title: [Solved] Let there be light
Post by: pmprog on July 06, 2010, 12:10:13 am
I've been playing around with the code
https://www.youtube.com/watch?v=VnQjsGGmeNk

SupSuper, a few things I noted when building this:

You're probably aware of these, but thought I'd mention if not  :D
Title: Re: Let there be light
Post by: SupSuper on July 06, 2010, 05:33:02 am
What, showing off without a hint of code??? How dare you! :P

Very nice work! ;D Looks a lot less gradual than the X-Com effect though, I presume you're not done yet?

I've been playing around with the code
https://www.youtube.com/watch?v=VnQjsGGmeNk

Quote
Variable Globe::_save is never populated (I added it to the constructor)
It's populated by Globe::setSavedGame.

Quote
Day to night palette appears to be 6 shades
I just tried it myself in the original, you're right that the ocean has 6 shades, but each land polygon seems to have 9-10 shades.
Title: Re: Let there be light
Post by: pmprog on July 06, 2010, 09:22:01 am
What, showing off without a hint of code??? How dare you! :P
Hehe, I was actually too embarressed to show the code, whilst the effect looked "okay", the code was pretty sloppy.

Very nice work! ;D Looks a lot less gradual than the X-Com effect though, I presume you're not done yet?
There are a number of issues with it at the moment, the main one being more than 1 sunrise in a day; secondly, I was getting the last longitude of each polygon and calculating a shade based of longitude and hour. This will also contribute to the relatively "jerky" sunrise.

It's populated by Globe::setSavedGame.
Good call (pardon the pun), I didn't see that  :)

I just tried it myself in the original, you're right that the ocean has 6 shades, but each land polygon seems to have 9-10 shades.
Hmm, perhaps I need to dump the palette out so I can see what I've got to work with. I'm sure if my "shade" variable went negative, or above 6, the globe went psychodellic.

Actually, I dumped the palette out:
(https://www.pmprog.co.uk/tmp/openxcompal.png)
At least 16 shades, which will also make a smoother transition.

On with the thinking cap.


Also, one last thing. As you noted that there are shades of ocean, you're just drawing a blue circle. Are there ocean polygons that are just not being drawn yet? or is that going to need some wizardry to paint?
Title: Re: Let there be light
Post by: SupSuper on July 06, 2010, 08:34:06 pm
Hmm, perhaps I need to dump the palette out so I can see what I've got to work with. I'm sure if my "shade" variable went negative, or above 6, the globe went psychodellic.

Actually, I dumped the palette out:
(https://www.pmprog.co.uk/tmp/openxcompal.png)
At least 16 shades, which will also make a smoother transition.
Well the real globe has actual textures (I haven't applied them because they cause terrible performance for some mysterious reason), which is probably why there's 16 shades of each colour but only 9-10 used for the day/light effect.

If necessary you can just have a min/max value to prevent the globe going psychedelic.

On with the thinking cap.


Also, one last thing. As you noted that there are shades of ocean, you're just drawing a blue circle. Are there ocean polygons that are just not being drawn yet? or is that going to need some wizardry to paint?
There are no ocean polygons (except in TFTD) stored in the game files. If I had to guess, the game might just generate a bunch of "ocean quads" itself that scroll horizontally along the flat map depending on the time, and get mapped as a "semi-sphere" on the globe. Or they use some wizardry involving circles or something, what do I know? :P

For future reference, I'd suggest checking out:
https://www.ufopaedia.org/index.php?title=WORLD.DAT
https://www.ufopaedia.org/index.php?title=PALETTES.DAT
Title: Re: Let there be light
Post by: pmprog on July 06, 2010, 11:13:50 pm
Well the real globe has actual textures (I haven't applied them because they cause terrible performance for some mysterious reason)
Yeah, totally forgot about them. In fact, I've been having a bit of a look at textures tonight, I can't seem to render them. They either don't draw anything, or crashes in the Surface::getPixel with _surface->pixels = 0x00000000. Odd.

The main reason I was looking was if there were different light levels per texture, but interestingly not, so you'd have to update the texture as your blitting it. Should make things a little more interesting. Either that, or as you read in TEXTURE.DAT, you create several SurfaceSets with the modified palettes to give your different light levels, then you can just blit from whichever source you need.

There are no ocean polygons (except in TFTD) stored in the game files. If I had to guess, the game might just generate a bunch of "ocean quads" itself that scroll horizontally along the flat map depending on the time, and get mapped as a "semi-sphere" on the globe. Or they use some wizardry involving circles or something, what do I know? :P
If it were me, I'd probably create a hardcoded full sphere polygon list (landscape can draw over the top), but this way you could texture the sea too if you wanted.
Title: Re: Let there be light
Post by: SupSuper on July 07, 2010, 01:38:22 am
Well the real globe has actual textures (I haven't applied them because they cause terrible performance for some mysterious reason)
Yeah, totally forgot about them. In fact, I've been having a bit of a look at textures tonight, I can't seem to render them. They either don't draw anything, or crashes in the Surface::getPixel with _surface->pixels = 0x00000000. Odd.
Enabling the poor texture rendering is an exercise in horror and shock and possibly frustration, but here goes:

1. Go to the Globe::draw function in Globe.cpp.
2. Comment out the filledPolygonColor(...) line and uncomment the texturedPolygon(...) line.
(if you're having fun you can also uncomment the polygonColor(...) line instead and see the globe in wireframe-o-vision!)

This would be enough, but because of how SDL works, surfaces need to be locked for pixel-level access and unlocked for blitting. So you also need to:

3. Comment out the lock() and unlock() lines so the blitting works.
4. Comment out the whole "base markers" drawing since anything with getPixel/setPixel will probably crash now since the surface is unlocked.
5. Comment out everything in Globe::blit except the Surface::blit(...) line so it doesn't redraw itself every few ticks (I'll optimize this eventually).

After all this you should have successfully revealed the beast:

(https://i6.photobucket.com/albums/y237/supsuper/OpenXcom/openxcom_horror.png)

Any attempt at rotation will take seconds and the textures don't even match the terrain and there's gaps in the polygons and oh gawd it's just so horrible that I just commented it out to never see the light of day while I cried in a corner for weeks and weeks :'( (seriously though with the huge performance problem there's no point in fixing the rest so I just moved on).

The main reason I was looking was if there were different light levels per texture, but interestingly not, so you'd have to update the texture as your blitting it. Should make things a little more interesting. Either that, or as you read in TEXTURE.DAT, you create several SurfaceSets with the modified palettes to give your different light levels, then you can just blit from whichever source you need.
Yeah palette wizardry is very common in X-Com, and since colors go from light to dark in the palette, you would just shift the palette a bit to the left for each polygon to make it darker. In OpenXcom you can achieve a similar effect with the Surface::offset function.

There are no ocean polygons (except in TFTD) stored in the game files. If I had to guess, the game might just generate a bunch of "ocean quads" itself that scroll horizontally along the flat map depending on the time, and get mapped as a "semi-sphere" on the globe. Or they use some wizardry involving circles or something, what do I know? :P
If it were me, I'd probably create a hardcoded full sphere polygon list (landscape can draw over the top), but this way you could texture the sea too if you wanted.
Yeah that would be the more sensible option today, but those were different times with only 32K of memory that you couldn't afford to waste or something who knows. :P Plus it makes the day/night effect look smoother over the ocean.
Title: Re: Let there be light
Post by: pmprog on July 07, 2010, 11:56:51 am
Enabling the poor texture rendering is an exercise in horror and shock and possibly frustration, but here goes:

1. Go to the Globe::draw function in Globe.cpp.
2. Comment out the filledPolygonColor(...) line and uncomment the texturedPolygon(...) line.
(if you're having fun you can also uncomment the polygonColor(...) line instead and see the globe in wireframe-o-vision!)

This would be enough, but because of how SDL works, surfaces need to be locked for pixel-level access and unlocked for blitting. So you also need to:

3. Comment out the lock() and unlock() lines so the blitting works.
4. Comment out the whole "base markers" drawing since anything with getPixel/setPixel will probably crash now since the surface is unlocked.
5. Comment out everything in Globe::blit except the Surface::blit(...) line so it doesn't redraw itself every few ticks (I'll optimize this eventually).
Actually, I did these instructions, it just left me with a blue globe (tho I might have forgot to change the surface lock). I'll have another look later

I can't see the image at work (it's blocked), but I'll fiddle with the code

Any attempt at rotation will take seconds and the textures don't even match the terrain and there's gaps in the polygons and oh gawd it's just so horrible that I just commented it out to never see the light of day while I cried in a corner for weeks and weeks :'( (seriously though with the huge performance problem there's no point in fixing the rest so I just moved on).
Hmmm, I might take a look at this, see if I can help out any, as I have a cunning plan how to perform this operation quickly...

Yeah palette wizardry is very common in X-Com, and since colors go from light to dark in the palette, you would just shift the palette a bit to the left for each polygon to make it darker. In OpenXcom you can achieve a similar effect with the Surface::offset function.
Thanks for the heads up on that.

Yeah that would be the more sensible option today, but those were different times with only 32K of memory that you couldn't afford to waste or something who knows. :P Plus it makes the day/night effect look smoother over the ocean.
Oh I knew why they didn't do it UFO, but I'm saying for OpenXcom you should. That said, TFTD had polygons for the seas, although I don't think they had as many for the land.
Title: Re: Let there be light
Post by: pmprog on July 09, 2010, 05:03:44 pm
https://www.pmprog.co.uk/tmp/xcomlight.swf

Code is in VB.NET for now (just checking my maths); Will convert to C++ later

Edit: SupSuper, if you think I'm "helping out" too much, let me know. I don't want you to feel like I'm railroading your project  ;)
Title: Re: Let there be light
Post by: SupSuper on July 09, 2010, 06:23:58 pm
https://www.pmprog.co.uk/tmp/xcomlight.swf

Code is in VB.NET for now (just checking my maths); Will convert to C++ later
I couldn't make much sense of the code, but the result looks alright. If you're making a light calculation based on time, something you should remember:
- Even though X-Com longitude goes from 0..360 (as opposed to -180..180), the GMT line is still at 0 longitude.
- Therefore, the "daylight" is centered at 0 at 12:00GMT, and the "nightlight" is centered at 0 at 0:00GMT (24-hour-format).

Edit: SupSuper, if you think I'm "helping out" too much, let me know. I don't want you to feel like I'm railroading your project  ;)
It's fine. As long as you keep working on the things I really don't feel like working on and would postpone for weeks (UGH GLOBES MATHS TRIGNOMETRIES THAT CAN WAIT), things get done faster. :P
Title: Re: Let there be light
Post by: pmprog on July 09, 2010, 09:53:41 pm
I couldn't make much sense of the code, but the result looks alright. If you're making a light calculation based on time, something you should remember:
- Even though X-Com longitude goes from 0..360 (as opposed to -180..180), the GMT line is still at 0 longitude.
- Therefore, the "daylight" is centered at 0 at 12:00GMT, and the "nightlight" is centered at 0 at 0:00GMT (24-hour-format).
That's fine, I had accounted for that.

It's fine. As long as you keep working on the things I really don't feel like working on and would postpone for weeks (UGH GLOBES MATHS TRIGNOMETRIES THAT CAN WAIT), things get done faster. :P
hehe, okay. I'll post a topic before I start looking at anything.

Oh, on a completely backwards note. we're going to have to undo a portion of the texture update. Whilst splitting out the globe and overlays might look cleaner anyway, the whole buffering of the globe was based on the fact that if you didn't rotate, it stayed the same. However, that completely neglected daylight rotation...


Here's the first look...
https://www.youtube.com/watch?v=JY5eJ_m2DlA

Edit: Attached patch
Title: Re: Let there be light
Post by: SupSuper on July 10, 2010, 04:10:41 pm
Oh, on a completely backwards note. we're going to have to undo a portion of the texture update. Whilst splitting out the globe and overlays might look cleaner anyway, the whole buffering of the globe was based on the fact that if you didn't rotate, it stayed the same. However, that completely neglected daylight rotation...
That's alright, it's still better this way since the globe still won't have to update every engine tick, but only every game tick, even if the difference is negligible. :P

Here's the first look...
https://www.youtube.com/watch?v=JY5eJ_m2DlA

Edit: Attached patch
Very nice! I'll integrate it later today if you're done tweaking it.
Title: Re: Let there be light
Post by: pmprog on July 10, 2010, 08:09:31 pm
Well, it still needs the ocean shading, and I think between texturing and lighting, there needs to be some optimisation; and I know exactly where to start...
Title: Re: Let there be light
Post by: pmprog on July 10, 2010, 10:44:40 pm
New patch.

Changes:

Title: Re: Let there be light
Post by: SupSuper on July 13, 2010, 05:32:16 am
I wouldn't worry too much about the accuracy of the light at this point, probably better to move on to more pressing issues first.

Also there's an SDL function for duplicating surfaces which is probably more efficient than just re-blitting it from scratch.


Edit: I commited your progress to the SVN. One thing to note, your "s" variable went all the way up to 8 while the array is only 0..7. ;) I increased the array to fix it.
Title: Re: Let there be light
Post by: pmprog on July 15, 2010, 09:25:33 pm
I wouldn't worry too much about the accuracy of the light at this point, probably better to move on to more pressing issues first.
Okaydoke, back to textures...

Also there's an SDL function for duplicating surfaces which is probably more efficient than just re-blitting it from scratch.
Ah, even better. I'll be honest, it's been a while since I properly played with SDL (and that was when I was looking to make my own XCOM style game)

Edit: I commited your progress to the SVN. One thing to note, your "s" variable went all the way up to 8 while the array is only 0..7. ;) I increased the array to fix it.
The s variable wasn't in patch 2. Hmmm, maybe I should have made it obvious that they are my patch revision numbers, not that you need to apply them both.

Title: Re: Let there be light
Post by: SupSuper on July 16, 2010, 03:47:10 am
I wouldn't worry too much about the accuracy of the light at this point, probably better to move on to more pressing issues first.
Okaydoke, back to textures...

Also there's an SDL function for duplicating surfaces which is probably more efficient than just re-blitting it from scratch.
Ah, even better. I'll be honest, it's been a while since I properly played with SDL (and that was when I was looking to make my own XCOM style game)

Edit: I commited your progress to the SVN. One thing to note, your "s" variable went all the way up to 8 while the array is only 0..7. ;) I increased the array to fix it.
The s variable wasn't in patch 2. Hmmm, maybe I should have made it obvious that they are my patch revision numbers, not that you need to apply them both.
I did only put in patch 2, here's the line:
Code: [Select]
s = (int)(( sin( (curTime * 2 * PI) + ((minLon + maxLon)/2) ) - 1 ) * -4);
Title: Re: Let there be light
Post by: pmprog on July 16, 2010, 01:58:47 pm
lol, yeah, ignore me  :-[
Title: Re: Let there be light
Post by: pmprog on July 20, 2010, 10:18:08 pm

Getting ever closer, still need to do the water. I'm also wondering if the south pole polygons are right, as the original seems to have plenty more polys there.

https://www.youtube.com/watch?v=8adPaE0pK1I
Title: Re: Let there be light
Post by: pmprog on July 21, 2010, 09:40:12 am
Finished

https://www.youtube.com/watch?v=Hnfhf-6xkM0

Title: Re: Let there be light
Post by: SupSuper on July 21, 2010, 08:31:36 pm

Getting ever closer, still need to do the water. I'm also wondering if the south pole polygons are right, as the original seems to have plenty more polys there.

https://www.youtube.com/watch?v=8adPaE0pK1I

It should be loaded exactly like the original. It might look different because due to the "spherical projection", the polygons on the poles end up overlapping each other and might come out differently depending on the order they're rendered in.

Finished

https://www.youtube.com/watch?v=Hnfhf-6xkM0


Wow! :o

I mean, really, wow, I'm glad it works but by now your code looks like nothing more than a huge blob to me. :P

Some criticism compared to the X-Com lighting:
- The shading between light/dark on the ocean is a lot wider and "unsmooth" than it is in X-Com.
- The shading in the poles seems a bit off, though I suppose they'll always look off. Theoretically, the poles should always be half-lit, and sometimes they seem to be full-lit/non-lit.

Btw, you don't need to declare every variable in the beginning of a block unless you actually plan to use it throughout the whole block. ;)
Title: Re: Let there be light
Post by: pmprog on July 21, 2010, 09:40:55 pm
It should be loaded exactly like the original. It might look different because due to the "spherical projection", the polygons on the poles end up overlapping each other and might come out differently depending on the order they're rendered in.
I tried reversing the rendering order although I don' remember much difference.

Wow! :o

I mean, really, wow, I'm glad it works but by now your code looks like nothing more than a huge blob to me. :P
Do you want me to tidy it up and comment it? It was kinda hack'n'slash last night; I'm not feeling 100% (think I've got a cold or something), so I've been really lethargic.

(https://www.pmprog.co.uk/tmp/openxcom-southpole.png)

- The shading between light/dark on the ocean is a lot wider and "unsmooth" than it is in X-Com.
That's pretty easy to tweak, just narrowing the bands will make it look smoother. In fact, I've done this for you in the attached patch

- The shading in the poles seems a bit off, though I suppose they'll always look off. Theoretically, the poles should always be half-lit, and sometimes they seem to be full-lit/non-lit.
This was why I was wondering if we were missing some polygons at the south pole. The North Pole looks okay. I'll be honest, I'm not really sure what else to do with the South Pole.

Btw, you don't need to declare every variable in the beginning of a block unless you actually plan to use it throughout the whole block. ;)
Sorry, force of habit. I've become accustomed to always declaring my variables at the top that I always do it this way, regardless of language

Oh, and if you move the backFace check before doing the 4 polarToCart functions, that'll speed you up a bit  ;)


Edit: Whilst I've refined one light/dark edge, that last patch screwed up the other side; and I don't know why (yet)
(https://www.pmprog.co.uk/tmp/openxcom-lightedge.png)

Edit 2: Fixed patch (5a), refined gradient more. Although the more you want to refine it, the more performance degradation you're going to get (at least using this algorithm as is)
Title: Re: Let there be light
Post by: SupSuper on July 22, 2010, 06:56:40 pm
Edit 2: Fixed patch (5a), refined gradient more. Although the more you want to refine it, the more performance degradation you're going to get (at least using this algorithm as is)
I'm guessing you're splitting the map into a huge series of quads and shading each accordingly, which makes sense in a 3D way, but not as much since this is simulated simplified 3D lighting. Most of the ocean will only have one shade.

My approach would be to simplify. At any point, the globe is half-day and half-night (ergo, half-brightest and half-darkest), so render that first:
(https://img837.imageshack.us/img837/4817/globedaynight.png)

Then apply the intermediate shades to the edges to smooth the transition:
(https://img690.imageshack.us/img690/931/globeshading.png)

And that's it. Then you just have to "shift" those polygons along the map according to the time (this should also make the effect look smoother without negative performance issues, as the "light" will actually move along the globe as opposed to just shifting shades in predefined quads). You might need to split them into multiple pieces depending on how you do the clipping (I presume you do clip them at the circle edges), but it should still turn out to be a lot less to render than now. Since the ocean is mostly flat and featureless, we should take advantage of it.

Mind you, I still wouldn't know how to implement this, so I'm just posting in case it gives you ideas to optimize your algorithm. :P
Title: Re: [Solved] Let there be light
Post by: pmprog on July 22, 2010, 09:38:38 pm
Unfortunately, it's not that simple, or at least from my primitive understanding of rotation and translation.

As you can see from this
(https://www.pmprog.co.uk/tmp/openxcom-lightquad.png)
You can't simply just draw the quad, because you'd effectively get no world.

To counter this, you have to calculate a number of points within the quad to get their 3D world x,y screen points. This gives you a 2D flat map of where you should fill.  This will be much faster than my version, however, this map however does not cater for front/back facing portions, and you will end up painting segments that should be on the hidden part of the sphere over the other.


That said, attached is a patch to speed things up a bit... and an explaination
Draw the circle as previous in either daylight or nighttime colours (nighttime is probably better), then once the sea shade has been calculated, if it is night (or day), skip the drawing of the polygon. This reduces the number of rendering calls made.

I'll be honest, if you're hoping for that really nicely drawn lines like the ones in the original UFO that renders quickly, you're going to need a smarter mathematician than I.
Title: Re: [Solved] Let there be light
Post by: SupSuper on July 23, 2010, 12:56:19 am
Sorry if I wasn't clear enough. I get that you can't just draw a huge quad across the globe. From what I gathered, your algorithm for rendering the ocean basically splits it into teeny tiny pieces and shades it accordingly with the help of a shades array (the images are not accurate, just really crude representations):

(https://img693.imageshack.us/img693/3206/globe1.png)

(if that's not how it works, then you can ignore the rest of it. :P)

So the shading is uniform across the map and easy to update. But if you need to make it smoother, you need to make every division smaller, incurring a performance problem:

(https://img638.imageshack.us/img638/5849/globe2.png)

So my thought was: most of the map is of the same shade (full lit or full dark):

(https://img34.imageshack.us/img34/1701/globe3.png)

So we take those constant shades, and split them into the biggest quads we can, kinda like you're already doing:

(https://img205.imageshack.us/img205/1343/globe4.png)

So we have the day/light effect, but the edges look ugly, obviously. But, we don't need to make the shading smoother across the whole map. We just need to smooth the edges. So we'd apply this separate, smaller, smoother in-between polygon shades just on the edges:

(https://img228.imageshack.us/img228/2227/globe5.png)

Then as time moved on, you could just move those tiny smooth-shaded polygons along as you change the shade on the bigger ones behind like you do now (or a mix, I'm not sure).

That way you could have smooth tiny polygons on the edges without impacting the whole map, because the map isn't this big long gradient. It's mostly the same two colors, with a small smoothing in-between them. Make sense?

If it doesn't then don't worry about, I don't really care too much about how nice or how efficient it is as long as it works. :P This is just the idea I had when I first thought of lighting the globe and I'd like to know if it actually has some merit.
Title: Re: [Solved] Let there be light
Post by: pmprog on July 23, 2010, 09:01:41 am
You're correct in understanding how it works (well, I no longer render the night sea as polygons as of patch 6), and I see what you are trying to say, but if you reduce the "resolution" of the solid areas by increasing the size of the quads, you will affect the edges of the globe, and that is the area where you really need the def, otherwise you get this
(https://www.pmprog.co.uk/tmp/openxcom-lightlodef.png)

One possible way I can think to speed it up, is after you've calculated the shade, if it's 0 (or you could group all the shades if you wanted too, remember there could be two sets of the intermediary shades), you add it to a "master polygon", and paint the polygon at the end, you would still need to process a refined set of points over the globe though
(https://www.pmprog.co.uk/tmp/openxcom-lightcomb.png)

Edit: Another way to speed it up is to reduce the tmpLon/tmpLat loops to the displayed hemisphere, then you don't even need to check for backfacing.
Title: Re: [Solved] Let there be light
Post by: SupSuper on July 23, 2010, 07:01:31 pm
Oh well, don't worry too much about my method then. You can keep on optimizing if you have nothing better to do, but otherwise it's not really worth wasting time on.

Anyways I've committed v6 to SVN so sync up when you can. Sorry for always tweaking your code to match my style, I just like keeping it all consistent. ;)
Title: Re: [Solved] Let there be light
Post by: pmprog on July 23, 2010, 11:07:01 pm
Oh well, don't worry too much about my method then. You can keep on optimizing if you have nothing better to do, but otherwise it's not really worth wasting time on.
If I think of a way, I will

Anyways I've committed v6 to SVN so sync up when you can. Sorry for always tweaking your code to match my style, I just like keeping it all consistent. ;)
lol, feel free to modify it any way you see fit