Mods and code changes.

[yrizoud]

I'm the one who has to apologize, you're new here and I have no idea what background you have    - I questioned your statements because you started about a Basescape "gameplay" feature, and somehow got sidetracked to changing the entire graphic layer - I didn't want to let you underestimate the tasks that you mention.

I understand barely half you describe of the fragment shader I got strong experience of CPU-bound coding and graphical applications, but my only actual experience with a GPU-bound API is by converting my SDL1 painting program to the SDL2 API with a DirectX back-end (and thus indexed paletted graphics into GPU textures in RGB color space - painting them is free, but drawing XOR lines is a nightmare).
I hope you didn't intend to use 8-bit paletted textures, I did some research recently and they are basically not supported on video cards. For example with OpenGL :
https://feedback.wildfiregames.com/report/opengl/feature/GL_EXT_paletted_texture
I heard many times of the "trick" of using a random 8bpp format (ex: greyscale) + a 256-pixel texture to store the lookup and a shader to do the conversion - but when I look for an actual case of somebody using it, I don't find any. I don't know if there's a catch.

So far, a strong argument for keeping the 8bit internal mode in OpenXcom is that it's supported on every SDL platform (which can range from a work/gamestation to a tiny 10-year-old handheld console). The resulting bitmap can be processed by any kinds of "rendering filters" - the default being SDL's own software renderer, which is the fastest I've seen to process a index->RGB video mode. Like, no problem even on a AGP x2 videocard of 1999.
Let this not discourage you from trying fun things with graphic cards, but you should know that the common trunk is going to keep what works for most platforms.

Do you intend your shader to try compute color distances (between current pixel and each of the palette colors), and keep the closest ? Distance in RGB cube is not accurate to humans' perception, and I fear it will be especially visible with UFO colors which are all very saturated. Color reduction is more an art than a science  . Even the following formula, which favors colors of same perceived lightness, should be more faithful : 26*(r1-r2)² + 55*(g1-g2)² + 19*(b1-b2)²

[Yankes]

I'm aware of the colour cycling that XCom uses, but it's not too difficult to emulate the effects in native 32bit. XCom seems to rarely, if ever, rely on complex cycling patterns. There are a few flashes and simple effects that are as trivial in 32bit as they are with a colour look up table.

It is not. When you switch from 8 to 32 you lose information that you can't restore. Main reason is that colors in palette can repeat. Because of that you can only approximate original behavior in RGB.
You can follow discussion about this there: https://openxcom.org/forum/index.php/topic,896.0.html

[RogerBennett]

Thanks for the reply -
I *have* just joined and figured adding scroll/zoom to the baseview would be a simple gameplay feature to wet my feet with rather than diving into something big. I can't easily do that using a blit on an 8-bit surface.

The next option is CLUT-to-RGBA, draw the basescape, and then RGBA-to-CLUT to output to the existing SDL_Surface. I figured that sort of thing is worth posting up because somebody might have already thought about it, already done it, or have another future use for it.
This looks like the minimum effort to get extended bases and base scrolling working. FWIW I did consider generating smaller zoom level versions of every tile in the base view, and using those with the indexed SDL blitter. This is exactly how we would have done it in the 90s and still an option, but it's a crappy option considering the asset cost vs having the colour transform code written and available for other things.

I don't much fancy reimplementing *everything* into 32-bit, and while I jumped the gun saying there isn't a good reason not to [especially so soon after joining], it would solve a lot of problems and provide a lot of flexibility.  The point of posting it here was to get feedback and awareness. I'd rather have replies than silence. Bottom line, I can't think of a technical reason not to change. You could start with pixel-for-pixel the same game, but would have options for improving the Base, Geo or Battle screens.

The GL_EXT_paletted_texture isn't worth it, and it solves a different problem really. And by the look of it isn't widely supported anyway. The process I described needs an RGB surface of any precision, and an 8-bit unsigned buffer for output - I'm literally performing the CLUT/RGBA conversions in GPU hardware using the same technique you'd do on the CPU ... just a lot faster.

I'd need programmable shaders, so I guess thats hardware from 2003 onwards. While this doesn't add a significant increase to the minimum spec, it does raise the bar into hardware rendering and will count out some devices.
Mobiles before OpenGL ES1.2 probably won't work, but that's all the way back to gingerbread on android support earlier than that was sketchy but the pre-gingerbread market penetration is tiny. If iOS is targeted it'd be fine on a 3G or higher. PCs from the Geforce FX/5 series onwards.
The system spec argument is a strong reason not to support new hardware, but it becomes a decision on *where* to draw the line since there is a line somewhere. It's also the sort of thing I'd be happier to discuss after the fact - get the feature working and then evaluate what the minimum spec is and if it's worth the tradeoff.

If hardware support was an issue for machines from the late-90's era, we could add a forward renderer using the OpenGL fixed function pipeline and/or software implementation. It's more work, and probably means cooking the assets into a file format for that platform... I'd rather just stick a minimum spec on the modpack but there are compromises we could make I'm sure. This would get us back to early Geforce 3, maybe earlier - TNTs even.
And even then, you could write a separate software renderer using shrunk assets for the bottom end platform(s), if it really came to the crunch and extended bases was deemed important enough to need to work on every platform.

I'm a gameplay guy, but I've had a little render/GPU experience and this looks easy enough and is on the critical path for the gameplay feature I'm focusing on (extending the baseview/basescape) that it was worth approaching as one feature.

My last concern is deviation from the source material. I enjoyed UFO a lot and too much change - even a different look and feel would probably put me off. It'd have to be a definite upgrade/improvement to really justify changing what is there in the core game. I think Mods and TCs might get more use out of extended bases.

Cheers, it's a lot to think about. I'll be able to hit the keyboard over the weekend with any luck.

[RogerBennett]

I don't think this topic
https://openxcom.org/forum/index.php/topic,896.0.html
Is relevant to what I'm saying.

It looks like they are talking about solving a different problem, with a different solution.  I've not looked at the battlescape renderer at all yet so I'm not in a very strong position to comment - but based on what was said in the thread it doesn't look relevant at all.

I'm not doing much more than reimplementing the SDL_Surface blit on the GPU with a 32-bit internal pipeline, so not changing colours, assets or shading.

[Yankes]

If you want modify only baseview then you can ignore all special effect from 8bit palette tricks. Because only battlescape and geoscape use it.
But then your solution will have small possible usage then. I usually prefer solution that can be used in more places that was primary designed for.

[RogerBennett]

Everything I'm openly planning is contained in the baseview, yes.
But the 32-8 and 8-32 conversion will be useful to have around, even just for reference.

With GPU rendering to 32-bit I don't have to lose info e.g. repeating colours. I've got an untouched 8bits that I could store a reverse-lookup for the CLUT for the entire framebuffer so you'd still know which pixel was which if you ever needed that info.  I can't think of a use case where i'd need it, but you could save it out to VRAM or even CPU addressable memory if you wanted it so the info isn't going to be lost.
More likely I'd bind a separate 8-bit render target for that, and either an RGB-24 for the colour buffer or a 32bit RGBA and use the alpha for transparency. Alpha is a little weird in an 8bit world so I'd been ignoring it.

I chose the baseview because it was easy, rather than starting with the Geoscape or Battlescape. It's self contained and the workload was low. Its a couple of hundred lines of code for the render. The colourspace conversion probably quadruples that, maybe more, but again that stuff could be useful.

If the Baseview was easy, I did have a peek at the geoscape because of the way the ugly way the world texture doesn't rotate with the world. I thought it'd be nice to fix that, but as soon as you start antialiasing you have the same RGBA-to-CLUT problems. Everything I thought of in the geoscape was cosmetic really. I thought about having clouds, waves, that sort of thing but they are just distractions.
Country highlights, clickable countries, and a few other options came to mind but nothing that grabbed me as code I wanted to tackle while I was focused on the Base.

The battlescape is a bit more of a beast. A 24 bit renderer could add transparent smoke, and variable zoom levels which are nice tweaks but not something I was too fussed over. It's too big to think about right now.

I did make a checklist of the work involved in using the BattlescapeGenerator to output a base defence mission as a vertex buffer object and then use a mesh of the base defence layout as the map screen in the BaseView. It'd be an interesting side project, generating a mesh isn't too hard, and you get nice pan, zoom, rotate on the BaseView map. It'd look cool and you'd be able to preview what the base defence would look like which might help some players, modders, or specific scenarios people came up with.
It was an interesting thought experiment, but there are a few problems to solve if you autogenerated meshes and I'm not sure I'd want to tackle those head on until I was a lot more familiar with the project.