Paletted 8bit -> 24bit shading.

[Daiky]

When using opengl, you can use shader programs to do the shading.
I tested openxcom with 24bit sprites in a very rough way, and it was painfully slow in software mode, so I assume it needs hardware acceleration anyway.

I know for example in Skyrim there are various mods that change the shader programs that darken textures during sunset,night and dawn to make them more realistic and less flat.

[Volutar]

Yankes, we cannot do that because of ambiguousnity. You must be reading my point not very carefully.

But how will you shade pixel with 24bit value (248,248,248) which have index 80 and index 224 (yes, two EXACT colors have different shading gradients in terms of palette) If you even manage to convert 24bit into 8bit palette (which will greatly cutoff colors), you will get ambiguous values, and that won't let you shade those sprites properly.

[Yankes]

yup, in my solution this couple of colors will merge, but rest of that gradient will stay maybe close to original (or will not, without test I only guessing )
one way to fix this is mark colors, we have 3 bytes with RGB and one byte with "gradient" that is used in my function to get correct shading (this 4th byte will be ignored by SDL).

[Volutar]

And how will you going to do those sprites with "palette" component???
That's the weirdest idea I ever heard

I do have an idea how we can convert old graphics and make new graphics "hued" in oldschool, paletted way. We should lower sprites light alittle, to get slightly off overlighted parts of gradient, so those tints will be there. But when drawing those sprites with light - we'll have not only darken them by particular formulas, but lighten as well (to get back to overlighted part).
Sortly - idea is to darken graphics before converting to 24bit, and light it back when drawing.

[Volutar]

I've made some tests of my idea.
Top line is palette.
Then "recreated" gradients, with light shift from -8  to -1(second light is taken as "main").

So as you can see, even this techniq won't get close to original, though is better than "flat shading" from first post (which is closest to bottom version).
In second shot I was using slightly different "lighting" method (while "shading" is the same).

Added: If we'll go into hardware (opengl), we can forget about styled shading
Found this https://en.wikipedia.org/wiki/Abney_effect

[karvanit]

The problem, as I understand it is the following:
The paletted image files contain the palette index for only one shade of the colour (colour group).
Right now we shade by moving inside this colour group, which contains the various shades.

Since we have the actual values of each shade for each group, we could plot the curve of shade, component and see if we can find Gamma that gives an approximate curve when used as pow(shade, component Gamma).

Shade and component are normalized to [0,1] (shade was [0,15] and component [0,255]).
I'm guessing that there should be a set of gamma functions that corresponds to these shades or that can approximate them good enough.

EDIT: You mean there are indices that are the same offset in different groups and have the same RGB value? And they are the original colour in the image (the data saved on file)? eg FAKE EXAMPLE to make it clear: Offset 0 in first group (warm wood) and 0 in third group (dark concrete) have the same RGB values and the image data contains a 0 in both cases? Because if that it the case then we have indeed lost information. But if it's not, then we may be able to get a good enough approximation.

[LCSand]

why not? this convertion can be done "on the fly". and then - HD sprites and graphics without palette limitation.

Short answer: Copyright
There are 2 ways to use the originals artwork.
1. Load it dynamically from the orignal data. I suppose that is what you mean by "on the fly".
2. Have an installer extract all the needed stuff from the original data only once.
The point is you can not distribute any non-original artwork.
And lets not start another discussion about copyright at this point, it is already discussed to death.

so you're suggesting to keep limited palette graphics? geez...

No you misunderstand me. Any converted artwork from the original will use only those indexed colors and as such can be shaded like before. The question is of course what happens once you have new artwork that actually uses new colors?

[Volutar]

Short answer: Copyright

I deny to discuss any of aspects of this. The real issue is TECHNICAL issue, and the only one. If you want to fall into copyright stuff- there's another topic for this.

I believe it's possible to overcome this 24bit shading, by using GLSL.

[SupSuper]

What's the point of trying to reproduce old palette shading in runtime if they're only gonna work with the old graphics and would just break again with new 24bpp graphics? You'll need to distinguish old and new graphics either way, so might as well just use a fragment shader to simulate palette effects or pre-generate all the lighted versions for old graphics.

Also the old graphics used VGA colors which only went up to (64,64,64), so the gradients probably aren't as crazy as they look converted to (256,256,256).

[Volutar]

SUpSUper, vga palette is just a pck convertion data, 6bit->8bit won't make any advantage. I'm totally agreed that "palette simulation" should work with ANY graphics, either old and new(added). That's why I've started this topic.. to gather and share ideas on how old palette shading(lighting) gradients could be simulated for non paletted 24/32bit sprites.

There's is a number of "hue ranges" in xcom1 which are getting more saturated when getting darker (more of deep red or blue in middle-bright part). I don't know how to simulate this, except for getting "default" sprites darkened to this middle-bright, oversaturated part, and then either getting sprite darker, or brighter. But when getting sprite brighter from this "middle-bright" we should not just multiply color components (that will lead to results shown in battlescape_palette2.png), but add some values (adding white color for brightening shown in battlescape_palette3.png). Though, it still far from original, since "oversaturated" (bright) parts aren't just "white" - they are cyan, or yellow, or orange...

It's possible to have sort of LUT, for significant bits of color components (for instance - 2 bits of red, 2 bits of green and 2 bits of blue, total 6 bits ->64 "brightening ways") so depending on what hue this color is closer, it will choose "overbright" color for it.. and it will work  (I hope) for both old graphics and new will match original palette style. Or at least will be more closer to it.

I'm not very familiar with hardware shading, but I believe it's possible to make this "fragment shader" which will be using this oversaturation and overbright trick fast way. But as fallback it will use flat shading without this palette magic.

[LCSand]

So you are trying to come up with a universally aplicable algorithm for shading in battlescape?
And the output should look as close to the original game shading as possible?

Well we can discuss a lot of theory, but as long as there is no testbed for the different algorithms we come up with so we can see the result(and performance), we can not realy judge wich algorithm is better.
And to realy test it out someone would have to create real 24bit artwork that uses more than the standart colors.

And does it realy have to look like the original shading?

[Volutar]

Not exact, but similar, with "oldschool" palette-like appearence, but of course HD and smoother.
I remember UFO:EU as game with special "night" appearence, which was not that nice in TFTD, and absolutely no such thing in Apocalypse. So, yes, I would like to come up with some sort of battlescape lightning algorithm (and why not glsl?), which will give nice result as for old-converted graphics, and for new graphics set as well.

[Daiky]

If you check out the TFTD Tactical palettes it is even more obvious:
https://www.ufopaedia.org/index.php?title=PALETTES.DAT

..how important the palette colors are. If you would just apply a flat shade, no matter if you are above or under water, you'll loose the underwater effect of the TFTD battlescapes.

[Volutar]

TFTD palette has blue tint addition for "middle" light levels. And of course it's quite essential for getting particular athmosphere. Some can say it can be done by blue light addition, but that will result in very blue screen, not just a tint.

[54x]

TFTD palette has blue tint addition for "middle" light levels. And of course it's quite essential for getting particular athmosphere. Some can say it can be done by blue light addition, but that will result in very blue screen, not just a tint.

Well unless you tint pixels blue in only certain light levels?