Messages

1

More Forks / Re: OXCE Lua/Vulkan

« on: February 18, 2025, 01:37:38 pm »

Just a quick update. I've finally managed to get most of the Vulkan stuff to a place I like. There's still some things I want to improve there, but I've got to make progress on the Lua side as well, so starting today I'm shifting focus to getting the Mod.cpp cut up and exposed through Lua.

The way things used to work has drastically changed. Before, you used to be able to write directly to surfaces, either by blitting or drawing directly on them. With Vulkan, you don't have that ability anymore. Everything is handled through primitives. When you create a primitive, a device local(GPU only) buffer is created and updated. The hardest primitive to make, by far, was the TextPrimitive(although an argument could be made for the line primitive since I did spend a week trying to find out why it was missing a single pixel).

Text wrapping, alignment, and placement is... complicated. Because of various localization needs, I ended up just using an established library called HarfBuzz for the text shaping, and yet another library called libunibreak for the word wrapping. The two combined should allow for languages like Hebrew and Arabic if the needs ever arises. Kerning is now a feature, which allows two character to adjust their overlap if that's something that is ever needed. However, something bothered me about the existing text rendering. Turns out, by using a special marker, you can change the font mid text, and fonts have multiple images associated with them. This messes up the render state which needs to track these changes and update the resource bindings. This meant I needed to implement Text Sections. While I was doing so, I decided that, well, since I have to support text sections, might as well see what else I can change about the text. That's when I decided to implement ANSI escape codes.

So, ANSI escape codes are a pain. I know. It would be nice if someone would just implement a CSS style processor that keeps track of a stack of states. However, the one really nice thing about ANSI escape codes is that I don't have to do that. I don't have to track a stack of states and manage all the details with the state changes. Instead, I can cut up the text into sections based on the escape code. If someone wants to write a stack based state management for text, feel free to do so. I had to make some choices as to where I wanted to spend my time. So, the cool thing is, now we have in-line text colors and styles. In a single line, you can have up to 16 colors. You can even change the background if you'd like(which will come in handy when it comes to text input selection). Cool, eh?

Here's the unit test running RGB colors through a single line of text:


And the code:
https://github.com/ken-noland/OpenXcom/blob/bd779c0f38a2d2eae8e056119e998f669cfa5bc3/src/Tests/Engine/Graphics/Primitives/TestText.cpp#L475

So, next up is Mod.cpp. Sorry, but that file is WAY too large and needs to be cut down. I'll be doing that with the use of Factories. I'm probably not going to do the whole thing, but pick and choose some of the rules that would get me the biggest results. After Mod.cpp, then I've got the input system to handle. That one gets a little complicated since I do want to have both Android and Windows/Mac/Linux all go through the same input interface.

2

More Forks / Re: OXCE Lua/Vulkan

« on: December 10, 2024, 01:35:05 am »

Hi Whispers,

I've starred the repo, and just wanted to register my interest in attempting to build your fork on macOS.

Thanks!

I have a "hackintosh" that I've created that I use to check compatibility. Specifically, I check to make sure it works with homebrew. I don't spend as much time on it as I do Linux/Android/Windows though, so I do expect things to break there from time to time.

3

More Forks / Re: OXCE Lua/Vulkan

« on: December 03, 2024, 01:17:53 am »

I thought I'd give a bit more of an update now that I've returned from a short "break". Unfortunately, I've had to pay some bills these last few weeks so I wasn't able to make much progress, but while I was working on other things, I've been thinking about how to handle the Vulkan renderer moving forward.

When working on something like this, the main goal is just to get it to work and then iterate on it. Well, I managed to get it to work. Sortof...

It was mimicking the surface blitting from SDL with some fairly basic shaders. I even went so far as to make the texture coordinates as integers instead of floats, using a fixed sampler, and then I was able to pass in the same coordinates from before and it would render the text, windows, and so on. Palettes were handled by uniform buffers and the palette ID was passed in as part of the rendering pipeline. I even went so far as to fix the aspect ratio in the game surface to windows surface shader.

However, the structure of it all bothered me. It wasn't very flexible and it felt a little like I was cramming functions into objects just for one single thing. That's not good design!

So, I've started cutting it up into smaller and smaller bits. While I'm at it, I want to make it API agnostic as well. If you want to render using DirectX12, that is something someone else can handle, but it should be possible.

Why go through all this effort? Easy! In the process of creating the new renderer, I've created a handful of "managers". Under normal circumstances, I don't like calling things "Managers". The term itself is ubiquitous since technically everything you write in code manages something, but it actually makes things super easy to manage resources. I'll hold my disdain for calling things Managers for now, and I'll just tell myself it sounds better than calling it "System". However, I digress. So, with that out of the way, practically every resource gets it's own data type and manager. All the Windows, Buttons, Text Boxes, Images, Palettes, Fonts and more! When I get to Battlescape, I'll be repeating the same pattern.

For instance, the base game only had 3 fonts that I know of. There's the DOS font that you see at the startup of the game while Mod.cpp loads, then there's the two in-game fonts(Large and Small). Now, I've got a FontManager. You can register your own font, complete with multiple pages and unicode support if you need it. Using the Large or Small font is still there, but now they are explicitly used by their handles. Also, I've kept the "turn big font into small font if it doesn't fit onscreen" for now, but I'm seriously debating if I should keep it in. Anyway, I digress. With the FontManager, I can now tie that into the entity inspector and scripting to allow me to register and use my own font if I want to.

However, one thing that struck me while I was doing the Window Manager is that I can now allow modders to create their own custom frames. The old way of drawing the frames was to use FillRect with preset values. With Vulkan, that doesn't work so well. I mean, you CAN do it, but I didn't want to. Vulkan works best if you create resources and then upload them to the GPU once, and only once. Draing the frame using FillRect is possible, but since the frame itself is animated when popping up, I didn't want to have to push up images every frame when I could just pass up coordinates. I can just upload a texture, specify the 8 cardinal and diagonal directions, then render those at the window coordinates. Instead of hardcoding that 5x5 corners, and the 5x1 sides, and 1x5 top and bottom, I can just let the WindowManager create the frame and the WindowFrame can be driven by the data. Data, as you can probably guess, can be modified through mods.

Why this long-winded explanation? Well, it's because in the process of redoing all the Window stuff with the new Vulkan images, I realized that a dedicated modder could use this framework to simply replace the border texture by specifying a file and the coordinates, then they can have ornate frames around in-game windows. Cool, eh? (I know, not the "killer feature" for this branch, but I thought it was nifty)

I'm going to spend a little while longer on Vulkan, mainly because I still have a week or two of real work to pay the power bills, but also because I'm having a bit of fun with it. I really enjoy trying to make the interfaces as elegant and clear as possible, and being distracted by real work has given me the opportunity to really just slowly refine what I have into something much cooler!

4

More Forks / Re: [1] Fork mod compatiblity reporting

« on: November 22, 2024, 01:53:05 pm »

I love this idea!

5

More Forks / Re: OXCE Lua/Vulkan

« on: November 12, 2024, 07:20:20 pm »

That's a theory, but how as a modder (read - game designer) that would help me? There should be super smart logic for procedural generation, if we do not use map blocks to generate battlescape map. I can't imagine any modder develop such a model, no matter if it would be Lua, C++, Pascal or whatever.
The only way I can imagine mapbgeneration can be fundamentally improved is what was done for XCOM 2 map generation with plots and parcels. I imagine, some sort of map block relation could be potentially implemented. Say, currently setting up something like road with parking attached to it is not trivial - you have to make complicated mapscrit with check, remove and then add commands to change original road tile to connect it with parking. I imagine there could be a more handy map generation process, but I can't see how Lua can help here, sorry. Am I wrong?

Good point! I totally get that serious procedural generation is more than just plugging in Lua—it’s about having the right tools and logic in place. Right now, the main goal with Lua is to make the current map block system as flexible as possible for modders, letting them tweak terrain and inject their own elements easily. It’s about getting up and running with what we already have, exposing what we can, and all hopefully without reinventing the wheel(too much).

But looking ahead, Lua could actually help with more rule-based or conditional generation if we go in that direction. Think of things like:

• Rule-based Layouts: Lua could allow modders to define placement rules, so you could script things like “place a road block next to open terrain” or “cluster buildings together.” This would let us build up a library of reusable conditions, similar to the parcels in XCOM 2.
• Dynamic Block Connections: Lua could enable blocks that adapt to their neighbors, making it easier to set up things like connected roads or parking lots without extensive manual scripting. It’s a way to add some fluidity to the current system without requiring everything to be hard-coded.
• Scriptable Templates: Imagine being able to design mini-layouts (like a courtyard, plaza, or forest cluster) that the generator pulls in based on the surrounding blocks or mission type. Lua could make it easy to slot these in dynamically.

So, while Lua isn’t the magic key to procedural generation alone, it gives us a framework to build on. And if we get the tools and logic right, it wouldn’t matter if the scripting was in Lua, C++, or even Pascal—the goal is really to expand the possibilities for map creation, whatever the language happens to be.

There are some tools we could look at in the future that are engine agnostic. An old colleague of mine has a tool for Unreal(and a version for Unity) called Apparance(https://www.apparance.uk/). It's a rule based parameterized procedural generator. Perhaps I'll pull him in to this conversation.

6

More Forks / Re: OXCE Lua/Vulkan

« on: November 11, 2024, 06:06:57 pm »

How?

So, the way Lua bindings are supposed to work is that each event in game is mapped to a corresponding event in Lua. Mods can then subscribe to those events, allowing them to alter or change things exposed through the event, but they can also modify or alter things that exist in the game itself, and that includes loading or creating assets and resources on-the-fly.

The way things are exposed is through Run Time Type Reflection.

So, if we are talking about just Battlescape Terrain, then you can see they are composed of Map Blocks. Each block could be created/loaded independently either through loading MCD and PCK files, or you could define them through Lua. Theoretically, because resources can be created and destroyed with Lua, then you can write your own handler when the terrain is generated which defines the map blocks per each terrain and that would allow you to create dynamic terrain, otherwise known as Procedural Terrain Generation.

By being able to intercept or override events in game and redirect them to scripts, and giving scripts the ability to create/load/unload resources on the fly, we hope to allow for much greater control for the mods.

7

More Forks / Re: OXCE Lua/Vulkan

« on: November 10, 2024, 02:15:31 am »

I've updated the main post and highlighted that this will not be going upstream. I'll keep posting updates here and leave the main post alone, but the old post felt like it needed to go.

8

More Forks / Re: OXCE Lua

« on: November 10, 2024, 01:20:05 am »

Moved.

I worry that by being here, people will expect OXCE to get this at some point.

Yeah, I can completely understand that. Things have drastically changed since I first posted, so I'll edit the main post to reflect that this is a fork, and will remain a fork unless that decision changes in the future.

9

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:57:13 am »

I have no idea to be honest. It doesn't quite fit with "WIP", as that's for mods AFAIK. It doesn't quite fit in "Build and Ports" either. Perhaps just in the root "OpenXcom Extended" board? I just worry that by being here, people will either expect Brutal to get this at some point, or that the branch will support Brutal in the future, which we don't really have plans for, and I've already been told that Ximli wants to keep things simple and not pull in branches that contain such drastic changes as this one introduces(unless it's proven solid and there's enough support for it)

10

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:38:03 am »

Also, I'm not sure BOXCE is the best place for this project. I mean, it really has nothing to do with Brutal at all, other than I forked from there.

I mean, if OXCE is "A younger brother of OpenXcom focused on modding support and QoL." then this would be like the step son from a previous marriage after OXCE has grown up

11

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:36:12 am »

I've posted an update:

https://github.com/ken-noland/OpenXcom/wiki/Update-%E2%80%90-November-9th-2024

In short...

What started as a simple Lua integration for OpenXcom turned into a major overhaul as each step uncovered more work. Moving to an Entity Component System (ECS) seemed like the best way to manage object lifecycles and interactions between Lua and C++, but it meant converting the game’s architecture piece by piece. This led to developing an entity inspector, which then revealed limitations with SDL’s event handling, pushing a switch to Vulkan. With SDL out of the picture, it made sense to revisit other areas—Options, Filesystem, Mods—that had grown cluttered over years of patching. Each fix led to another realization, eventually bringing about the need for a centralized Engine class to handle all the stuff SDL used to do, but outside of the Game class.

Switching to Vulkan and the ECS framework means resources are now managed through handles. With new Options class, a cleaner filesystem, and a CMake-based build system, the project is more cross-platform ready. The end goal is still to provide a more open foundation for modders, allowing Lua to prototype features that could later be optimized in C++. It has definitely taken longer than planned, but is still progressing.

I'm currently working on the oxce-lua-vulkan branch. I'll try to post updates more regularly now that the core is almost done.

12

Programming / Re: SDL2 branch status?

« on: November 09, 2024, 02:41:41 am »

I think it's fair to say that it is abandoned at this point. SDL1 is no longer offered on Mac and has been pulled from homebrew, and I didn't waste much time trying to find if I could download it elsewhere.

13

Brutal AI / Re: BOXCE for Windows XP

« on: November 09, 2024, 02:35:24 am »

wait, people still run the game on WinXP? Serious question, as I'm working on something that would definitely not work on XP.

14

OpenXcom Extended (OXCE) / Re: Converting OXCE to rapidyaml

« on: November 09, 2024, 02:29:11 am »

Hmmm... maybe worth knowing, but maybe not... I've switched a lot of the resource loading and unloading to Json for the Lua/Vulkan branch. Granted, it's not a released branch and I'm still a fair ways away from releasing it, but I thought it might be worth mentioning.

The way I deal with save game, mod loading(rulesets and so on) and config stuff is through serializers which are either hand crafted or auto-generated from the run time type information. What this means is that there is a considerable increase in compile times and executable size due in large part to the compilation and linking of a code generator which then needs further compilation, but saves time on loading during runtime. For Lua and Json, all the bindings are automatic, but since a goal of the Lua/Vulkan project is to retain backwards compatibility, then I manually put in the YAML serializers so they conform to the original OXCE rules. In fact, it makes it nicer because I can put whatever I want in the Json and not have to worry if I might break some mod out there that someone wrote a long time ago. Granted, I've only made it as far as the config stuff at the moment(config, localization, and a few other bits), so I might run into some issue down the line which might make it more of a problem, but working with Json in debug is so much faster than YAML. However, equally, it is worth mentioning that I barely notice YAML in release mode.

Use this information however you see fit. I'd be happy to see SOMETHING improve debug loading and unloading of mods. The Lua/Vulkan branch is really broken at the moment and I won't make any promises as to when it will be released, so there's also that to consider

15

More Forks / Re: OXCE Lua

« on: October 21, 2024, 01:21:29 pm »

Yet another update.

I decided to use run time type reflection to make the Lua binding easier, and in the process, I created the entity inspector. Early work on the entity inspector had it running in the same process as the game and using wxWidgets to manage the UI elements. Basically, anything that is an entity can have all it's components and their properties viewed and possibly edited by the inspector. However, I ran into a significant snag. It turns out that SDL hogs all the input for the process and there is no way for SDL and wxWidgets to live happily side by side in the same process. So, I decided to ditch SDL and rewrite the audio/video/input systems.

I've got it to the point now where most of the UI is rendering using Vulkan, and audio is through OpenAL, but that can be swapped out later if needed. I have a lot of work to do on the video playback, but most of the surface level stuff is functional.

I've ripped out the file handling and options logic and replaced it with a virtual filesystem and new options class because, well, to be honest, it was bugging me having things stored as global variables.

Everything I've done so far has been compiled on Windows and Linux. I don't have a Mac, although I did spend some time to see if I could spin up a Mac VMWare instance. That does appear to be working, but I haven't had a chance to get it all set up with homebrew and whatever else is needed. I did ditch all build systems other than CMake, so I'm not going to maintain the visual studio solution files or makefiles or any of that anymore. This is not a reflection on those build systems, but CMake can target almost every major IDE and handle the prerequisite gathering and all of that stuff without having to try to locate the right libraries for your system and place them in the right folder.

So, what I thought would be a few weeks of "just wrap things in Lua" turned out to be a major rewrite on all the core systems. I even changed up the log system so I could, in theory, have an in game console outside of the entity inspector. I've probably broken a few things along the way, but the general direction is good and seeing the front end menu pop up after weeks of uncertainty, well, really gave me a huge boost of morale.