OXCE Lua/Vulkan

[Whispers]

Hey,

That's right, I've started it.

https://github.com/ken-noland/OpenXcom

I’m excited to introduce you to my ongoing OpenXcom fork, which started as a fork of Brutal (BOXCE) but has now evolved into a distinct project with its own vision and goals. While this project won’t merge with OpenXcom Extended (OXCE), the aim is to open new doors for modders by providing a foundation that’s as flexible and accessible as possible. You can learn more about OXCE here to see where this project diverges.

Project Goals

This project’s primary goal is to enable a level of scripting that hasn’t been possible before, using Lua—a scripting language widely recognized and commonly used in the modding community. Lua will allow modders to script almost everything in the game, from region behaviors and globe mechanics to detailed particle effects, providing an unprecedented level of customization. My hope is that Lua scripting will make OpenXcom modding accessible to a broader audience and allow for rapid prototyping and development without diving into the complexities of the underlying codebase.

As part of this overhaul, we are also upgrading the underlying systems to modern APIs by replacing SDL with Vulkan for graphics and OpenAL for audio. This update ensures that OpenXcom remains compatible with modern platforms and continues to be maintainable long-term.

Building Toward New Possibilities

With this new scripting system, there are some powerful modding possibilities within reach:

• Complete UI Overhauls: The scripting system allows modders to redefine the game’s UI, enabling complete overhauls if desired. Modders can customize surfaces, blocks, and interactive elements, creating tailored interfaces.
  
• Procedural Terrain Generation: Modders can define their own terrain elements and structures, potentially enabling procedural terrain for dynamic mission environments.
  
• Custom Attributes: Lua scripting supports defining custom attributes in any object, allowing you to add new stats or attributes without the need for complicated workarounds
  
• Dynamic Mission Types and Objectives: With scripting, modders could introduce custom mission types, adapting objectives based on player actions or randomized conditions.
  
• Custom AI Behaviors: Lua makes it possible to script custom AI tactics, from enemy behaviors to ally actions, offering more reactive and intelligent opponents.
  
• Environmental Effects and Weather Systems: Modders could add dynamic environmental conditions, such as weather or hazardous zones, that impact gameplay in real time.
  
• Campaign Progression and Branching Paths: The system could support dynamic campaigns with branching storylines, influenced by player choices and mission outcomes.
  
• Advanced Resource Management: Beyond custom attributes, modders could introduce complex resource systems, alternate economies, or even faction loyalty mechanics.
  
• Enhanced Mod Compatibility and Interactions: The Lua foundation allows for dynamic interaction between mods, creating new possibilities for layered, collaborative mod features.
  

Why Lua and Vulkan?

The choice to use Lua for scripting was driven by the desire to make OpenXcom modding as accessible and flexible as possible. Lua is a widely-used language with a low learning curve, making it easy for modders to jump in and create complex, game-changing modifications without needing to delve into C++ code. Its simplicity is paired with powerful functionality, allowing modders to define game behaviors, create custom attributes, and introduce unique gameplay mechanics. Lua also integrates smoothly with C++, making it an ideal choice for a project where scripts need to interact closely with the game’s underlying systems.

The switch from SDL to Vulkan was equally intentional. SDL has served the project well for years, but Vulkan offers greater control over graphics rendering and better performance on modern hardware. This control is critical in allowing OpenXcom to support advanced visual features and to run efficiently across platforms, including potential future extensions to Mac and Linux. Vulkan also supports multi-threading better than SDL, which is a huge advantage for performance as the game grows more complex. This shift paves the way for visual enhancements, optimized performance, and cross-platform support, ensuring OpenXcom is equipped to handle both the current goals and future expansions.

Runtime Type Generation and Serialization

A key feature in this project is the use of runtime type generation to handle serialization between the engine, JSON objects on disk, and Lua functions and tables. Using runtime type information, the system automatically generates code to serialize and deserialize data to and from the engine. This allows modders to define custom attributes directly in Lua, which can then be accessed within the engine or stored in JSON files for persistence. This approach reduces the need for extensive C++ modifications for each new attribute or entity, simplifying the integration of modder-defined data.

This setup relies on type reflection to dynamically retrieve type information and automate serialization code generation. Custom attributes or entities defined in Lua are accessible within the engine, with JSON handling save states and configurations without requiring a unique serializer for each type. By generating Lua and JSON serializers automatically, components—such as gameplay attributes, UI elements, or terrain features—are consistently accessible across scripting and storage.

To maintain backward compatibility with older mods, YAML serialization is handled through hand-coded serializers. This approach ensures that legacy mods developed with YAML remain functional without modification. The combination of runtime type generation, automatic Lua and JSON serialization, and manual YAML support provides a flexible system that accommodates both legacy and new modding capabilities.

Future Vision

Looking beyond the initial goals, this project opens up some ambitious possibilities for the future of OpenXcom modding. Here are a few of the ideas that could become feasible with the new foundation:

• Playing X-COM 1 and 2 in a Seamless Campaign: Imagine starting with soldiers from X-COM 1 and transitioning directly into X-COM 2, carrying over your personnel, resources, and accomplishments to fight new threats with the same team.
  
• Complex Geopolitical Dynamics: The new scripting system could allow for deeper political interactions, where countries and factions have evolving allegiances, conflicts, and responses to player actions. Modders could build complex systems of geopolitical intrigue, making each decision impact the broader world in meaningful ways.
  
• Fog of War: Enabling an ongoing fog of war across the global map would add a layer of mystery and tension to strategic decision-making, concealing alien bases, research installations, and potentially hostile factions until they’re scouted or discovered.
  
• Multiple Globes for Interplanetary Operations: This new foundation makes it feasible to support multiple globe views, allowing interplanetary missions where players can explore, engage, and defend across different planets, each with its own conditions and alien threats. An interplanetary scope would expand the tactical and strategic gameplay far beyond Earth.
  

With these possibilities, OpenXcom could evolve into a platform that accommodates far more complex, immersive, and expansive gameplay. For anyone interested in following or contributing to this vision, I’ve outlined the project roadmap on the project’s GitHub Wiki. While we’re not yet at a public release stage, nightly builds are planned as we make further progress.

Thank you to everyone following and supporting the project. This is still a work in progress, but I’m excited to see what new experiences this foundation will enable in the OpenXcom universe. Stay tuned for more updates as we continue expanding OpenXcom’s possibilities!

[Whispers]

Update:

Right, there's been some big modifications to the original source. Maxmahem has graciously offered to help move towards a more Entity Component System compliant code base. The reason why is that we need handles to objects so we don't have to manage the object lifecycle as much on the Lua side. I've started working on the UI side, wrapping all the interface objects and converting those over to composable elements(components) to make things easier when translating to and from Lua.

Overall, this is shaping up to be a huge refactor.

I have a branch that I've been toying with where I take the Lua bindings and I redirect it to a json serializer. I open up a plain old TCP socket and I accept basic HTTP GET and POST commands which basically means I can now edit anything that is exposed to Lua through a web page. It's actually something I've done for a lot of projects I've worked on, and it greatly helps test and diagnose issues. I'll probably merge that in to the Lua branch soon, although it has very little to do with Lua.

Would love to get your thoughts on these changes. Feel free to leave feedback here, or on the repo, any time!

[Whispers]

Yet another update.

I decided to use run time type reflection to make the Lua binding easier, and in the process, I created the entity inspector. Early work on the entity inspector had it running in the same process as the game and using wxWidgets to manage the UI elements. Basically, anything that is an entity can have all it's components and their properties viewed and possibly edited by the inspector. However, I ran into a significant snag. It turns out that SDL hogs all the input for the process and there is no way for SDL and wxWidgets to live happily side by side in the same process. So, I decided to ditch SDL and rewrite the audio/video/input systems.

I've got it to the point now where most of the UI is rendering using Vulkan, and audio is through OpenAL, but that can be swapped out later if needed. I have a lot of work to do on the video playback, but most of the surface level stuff is functional.

I've ripped out the file handling and options logic and replaced it with a virtual filesystem and new options class because, well, to be honest, it was bugging me having things stored as global variables.

Everything I've done so far has been compiled on Windows and Linux. I don't have a Mac, although I did spend some time to see if I could spin up a Mac VMWare instance. That does appear to be working, but I haven't had a chance to get it all set up with homebrew and whatever else is needed. I did ditch all build systems other than CMake, so I'm not going to maintain the visual studio solution files or makefiles or any of that anymore. This is not a reflection on those build systems, but CMake can target almost every major IDE and handle the prerequisite gathering and all of that stuff without having to try to locate the right libraries for your system and place them in the right folder.

So, what I thought would be a few weeks of "just wrap things in Lua" turned out to be a major rewrite on all the core systems. I even changed up the log system so I could, in theory, have an in game console outside of the entity inspector. I've probably broken a few things along the way, but the general direction is good and seeing the front end menu pop up after weeks of uncertainty, well, really gave me a huge boost of morale.

[Whispers]

I've posted an update:

https://github.com/ken-noland/OpenXcom/wiki/Update-%E2%80%90-November-9th-2024

In short...

What started as a simple Lua integration for OpenXcom turned into a major overhaul as each step uncovered more work. Moving to an Entity Component System (ECS) seemed like the best way to manage object lifecycles and interactions between Lua and C++, but it meant converting the game’s architecture piece by piece. This led to developing an entity inspector, which then revealed limitations with SDL’s event handling, pushing a switch to Vulkan. With SDL out of the picture, it made sense to revisit other areas—Options, Filesystem, Mods—that had grown cluttered over years of patching. Each fix led to another realization, eventually bringing about the need for a centralized Engine class to handle all the stuff SDL used to do, but outside of the Game class.

Switching to Vulkan and the ECS framework means resources are now managed through handles. With new Options class, a cleaner filesystem, and a CMake-based build system, the project is more cross-platform ready. The end goal is still to provide a more open foundation for modders, allowing Lua to prototype features that could later be optimized in C++. It has definitely taken longer than planned, but is still progressing.

I'm currently working on the oxce-lua-vulkan branch. I'll try to post updates more regularly now that the core is almost done.

[Whispers]

Also, I'm not sure BOXCE is the best place for this project. I mean, it really has nothing to do with Brutal at all, other than I forked from there.

I mean, if OXCE is "A younger brother of OpenXcom focused on modding support and QoL." then this would be like the step son from a previous marriage after OXCE has grown up

[Meridian]

where should I move it then?

[Whispers]

I have no idea to be honest. It doesn't quite fit with "WIP", as that's for mods AFAIK. It doesn't quite fit in "Build and Ports" either. Perhaps just in the root "OpenXcom Extended" board? I just worry that by being here, people will either expect Brutal to get this at some point, or that the branch will support Brutal in the future, which we don't really have plans for, and I've already been told that Ximli wants to keep things simple and not pull in branches that contain such drastic changes as this one introduces(unless it's proven solid and there's enough support for it)

[Meridian]

Perhaps just in the root "OpenXcom Extended" board?

Moved.

I just worry that by being here, people will either expect Brutal to get this at some point, or that the branch will support Brutal in the future, which we don't really have plans for, and I've already been told that Ximli wants to keep things simple and not pull in branches that contain such drastic changes as this one introduces(unless it's proven solid and there's enough support for it)

I worry that by being here, people will expect OXCE to get this at some point.

[Whispers]

Moved.

I worry that by being here, people will expect OXCE to get this at some point.

Yeah, I can completely understand that. Things have drastically changed since I first posted, so I'll edit the main post to reflect that this is a fork, and will remain a fork unless that decision changes in the future.

[Whispers]

I've updated the main post and highlighted that this will not be going upstream. I'll keep posting updates here and leave the main post alone, but the old post felt like it needed to go.

[Meridian]

Thanks for the update.
I have also deleted a few early comments, which were now out of context and irrelevant.

PS: also please update the typo in the first post: OXCE stands for OpenXcom Extended, not OpenXcom Community Edition

[Finnik]

Procedural Terrain Generation: Modders can define their own terrain elements and structures, potentially enabling procedural terrain for dynamic mission environments.

How?

[Whispers]

How?

So, the way Lua bindings are supposed to work is that each event in game is mapped to a corresponding event in Lua. Mods can then subscribe to those events, allowing them to alter or change things exposed through the event, but they can also modify or alter things that exist in the game itself, and that includes loading or creating assets and resources on-the-fly.

The way things are exposed is through Run Time Type Reflection.

So, if we are talking about just Battlescape Terrain, then you can see they are composed of Map Blocks. Each block could be created/loaded independently either through loading MCD and PCK files, or you could define them through Lua. Theoretically, because resources can be created and destroyed with Lua, then you can write your own handler when the terrain is generated which defines the map blocks per each terrain and that would allow you to create dynamic terrain, otherwise known as Procedural Terrain Generation.

By being able to intercept or override events in game and redirect them to scripts, and giving scripts the ability to create/load/unload resources on the fly, we hope to allow for much greater control for the mods.

[Finnik]

That's a theory, but how as a modder (read - game designer) that would help me? There should be super smart logic for procedural generation, if we do not use map blocks to generate battlescape map. I can't imagine any modder develop such a model, no matter if it would be Lua, C++, Pascal or whatever.
The only way I can imagine mapbgeneration can be fundamentally improved is what was done for XCOM 2 map generation with plots and parcels. I imagine, some sort of map block relation could be potentially implemented. Say, currently setting up something like road with parking attached to it is not trivial - you have to make complicated mapscrit with check, remove and then add commands to change original road tile to connect it with parking. I imagine there could be a more handy map generation process, but I can't see how Lua can help here, sorry. Am I wrong?

[Whispers]

That's a theory, but how as a modder (read - game designer) that would help me? There should be super smart logic for procedural generation, if we do not use map blocks to generate battlescape map. I can't imagine any modder develop such a model, no matter if it would be Lua, C++, Pascal or whatever.
The only way I can imagine mapbgeneration can be fundamentally improved is what was done for XCOM 2 map generation with plots and parcels. I imagine, some sort of map block relation could be potentially implemented. Say, currently setting up something like road with parking attached to it is not trivial - you have to make complicated mapscrit with check, remove and then add commands to change original road tile to connect it with parking. I imagine there could be a more handy map generation process, but I can't see how Lua can help here, sorry. Am I wrong?

Good point! I totally get that serious procedural generation is more than just plugging in Lua—it’s about having the right tools and logic in place. Right now, the main goal with Lua is to make the current map block system as flexible as possible for modders, letting them tweak terrain and inject their own elements easily. It’s about getting up and running with what we already have, exposing what we can, and all hopefully without reinventing the wheel(too much).

But looking ahead, Lua could actually help with more rule-based or conditional generation if we go in that direction. Think of things like:

• Rule-based Layouts: Lua could allow modders to define placement rules, so you could script things like “place a road block next to open terrain” or “cluster buildings together.” This would let us build up a library of reusable conditions, similar to the parcels in XCOM 2.
• Dynamic Block Connections: Lua could enable blocks that adapt to their neighbors, making it easier to set up things like connected roads or parking lots without extensive manual scripting. It’s a way to add some fluidity to the current system without requiring everything to be hard-coded.
• Scriptable Templates: Imagine being able to design mini-layouts (like a courtyard, plaza, or forest cluster) that the generator pulls in based on the surrounding blocks or mission type. Lua could make it easy to slot these in dynamically.

So, while Lua isn’t the magic key to procedural generation alone, it gives us a framework to build on. And if we get the tools and logic right, it wouldn’t matter if the scripting was in Lua, C++, or even Pascal—the goal is really to expand the possibilities for map creation, whatever the language happens to be.

There are some tools we could look at in the future that are engine agnostic. An old colleague of mine has a tool for Unreal(and a version for Unity) called Apparance(https://www.apparance.uk/). It's a rule based parameterized procedural generator. Perhaps I'll pull him in to this conversation.