Messages

1

More Forks / Re: OXCE Lua/Vulkan

« on: December 03, 2024, 01:17:53 am »

I thought I'd give a bit more of an update now that I've returned from a short "break". Unfortunately, I've had to pay some bills these last few weeks so I wasn't able to make much progress, but while I was working on other things, I've been thinking about how to handle the Vulkan renderer moving forward.

When working on something like this, the main goal is just to get it to work and then iterate on it. Well, I managed to get it to work. Sortof...

It was mimicking the surface blitting from SDL with some fairly basic shaders. I even went so far as to make the texture coordinates as integers instead of floats, using a fixed sampler, and then I was able to pass in the same coordinates from before and it would render the text, windows, and so on. Palettes were handled by uniform buffers and the palette ID was passed in as part of the rendering pipeline. I even went so far as to fix the aspect ratio in the game surface to windows surface shader.

However, the structure of it all bothered me. It wasn't very flexible and it felt a little like I was cramming functions into objects just for one single thing. That's not good design!

So, I've started cutting it up into smaller and smaller bits. While I'm at it, I want to make it API agnostic as well. If you want to render using DirectX12, that is something someone else can handle, but it should be possible.

Why go through all this effort? Easy! In the process of creating the new renderer, I've created a handful of "managers". Under normal circumstances, I don't like calling things "Managers". The term itself is ubiquitous since technically everything you write in code manages something, but it actually makes things super easy to manage resources. I'll hold my disdain for calling things Managers for now, and I'll just tell myself it sounds better than calling it "System". However, I digress. So, with that out of the way, practically every resource gets it's own data type and manager. All the Windows, Buttons, Text Boxes, Images, Palettes, Fonts and more! When I get to Battlescape, I'll be repeating the same pattern.

For instance, the base game only had 3 fonts that I know of. There's the DOS font that you see at the startup of the game while Mod.cpp loads, then there's the two in-game fonts(Large and Small). Now, I've got a FontManager. You can register your own font, complete with multiple pages and unicode support if you need it. Using the Large or Small font is still there, but now they are explicitly used by their handles. Also, I've kept the "turn big font into small font if it doesn't fit onscreen" for now, but I'm seriously debating if I should keep it in. Anyway, I digress. With the FontManager, I can now tie that into the entity inspector and scripting to allow me to register and use my own font if I want to.

However, one thing that struck me while I was doing the Window Manager is that I can now allow modders to create their own custom frames. The old way of drawing the frames was to use FillRect with preset values. With Vulkan, that doesn't work so well. I mean, you CAN do it, but I didn't want to. Vulkan works best if you create resources and then upload them to the GPU once, and only once. Draing the frame using FillRect is possible, but since the frame itself is animated when popping up, I didn't want to have to push up images every frame when I could just pass up coordinates. I can just upload a texture, specify the 8 cardinal and diagonal directions, then render those at the window coordinates. Instead of hardcoding that 5x5 corners, and the 5x1 sides, and 1x5 top and bottom, I can just let the WindowManager create the frame and the WindowFrame can be driven by the data. Data, as you can probably guess, can be modified through mods.

Why this long-winded explanation? Well, it's because in the process of redoing all the Window stuff with the new Vulkan images, I realized that a dedicated modder could use this framework to simply replace the border texture by specifying a file and the coordinates, then they can have ornate frames around in-game windows. Cool, eh? (I know, not the "killer feature" for this branch, but I thought it was nifty)

I'm going to spend a little while longer on Vulkan, mainly because I still have a week or two of real work to pay the power bills, but also because I'm having a bit of fun with it. I really enjoy trying to make the interfaces as elegant and clear as possible, and being distracted by real work has given me the opportunity to really just slowly refine what I have into something much cooler!

2

More Forks / Re: [1] Fork mod compatiblity reporting

« on: November 22, 2024, 01:53:05 pm »

I love this idea!

3

More Forks / Re: OXCE Lua/Vulkan

« on: November 12, 2024, 07:20:20 pm »

That's a theory, but how as a modder (read - game designer) that would help me? There should be super smart logic for procedural generation, if we do not use map blocks to generate battlescape map. I can't imagine any modder develop such a model, no matter if it would be Lua, C++, Pascal or whatever.
The only way I can imagine mapbgeneration can be fundamentally improved is what was done for XCOM 2 map generation with plots and parcels. I imagine, some sort of map block relation could be potentially implemented. Say, currently setting up something like road with parking attached to it is not trivial - you have to make complicated mapscrit with check, remove and then add commands to change original road tile to connect it with parking. I imagine there could be a more handy map generation process, but I can't see how Lua can help here, sorry. Am I wrong?

Good point! I totally get that serious procedural generation is more than just plugging in Lua—it’s about having the right tools and logic in place. Right now, the main goal with Lua is to make the current map block system as flexible as possible for modders, letting them tweak terrain and inject their own elements easily. It’s about getting up and running with what we already have, exposing what we can, and all hopefully without reinventing the wheel(too much).

But looking ahead, Lua could actually help with more rule-based or conditional generation if we go in that direction. Think of things like:

• Rule-based Layouts: Lua could allow modders to define placement rules, so you could script things like “place a road block next to open terrain” or “cluster buildings together.” This would let us build up a library of reusable conditions, similar to the parcels in XCOM 2.
• Dynamic Block Connections: Lua could enable blocks that adapt to their neighbors, making it easier to set up things like connected roads or parking lots without extensive manual scripting. It’s a way to add some fluidity to the current system without requiring everything to be hard-coded.
• Scriptable Templates: Imagine being able to design mini-layouts (like a courtyard, plaza, or forest cluster) that the generator pulls in based on the surrounding blocks or mission type. Lua could make it easy to slot these in dynamically.

So, while Lua isn’t the magic key to procedural generation alone, it gives us a framework to build on. And if we get the tools and logic right, it wouldn’t matter if the scripting was in Lua, C++, or even Pascal—the goal is really to expand the possibilities for map creation, whatever the language happens to be.

There are some tools we could look at in the future that are engine agnostic. An old colleague of mine has a tool for Unreal(and a version for Unity) called Apparance(https://www.apparance.uk/). It's a rule based parameterized procedural generator. Perhaps I'll pull him in to this conversation.

4

More Forks / Re: OXCE Lua/Vulkan

« on: November 11, 2024, 06:06:57 pm »

How?

So, the way Lua bindings are supposed to work is that each event in game is mapped to a corresponding event in Lua. Mods can then subscribe to those events, allowing them to alter or change things exposed through the event, but they can also modify or alter things that exist in the game itself, and that includes loading or creating assets and resources on-the-fly.

The way things are exposed is through Run Time Type Reflection.

So, if we are talking about just Battlescape Terrain, then you can see they are composed of Map Blocks. Each block could be created/loaded independently either through loading MCD and PCK files, or you could define them through Lua. Theoretically, because resources can be created and destroyed with Lua, then you can write your own handler when the terrain is generated which defines the map blocks per each terrain and that would allow you to create dynamic terrain, otherwise known as Procedural Terrain Generation.

By being able to intercept or override events in game and redirect them to scripts, and giving scripts the ability to create/load/unload resources on the fly, we hope to allow for much greater control for the mods.

5

More Forks / Re: OXCE Lua/Vulkan

« on: November 10, 2024, 02:15:31 am »

I've updated the main post and highlighted that this will not be going upstream. I'll keep posting updates here and leave the main post alone, but the old post felt like it needed to go.

6

More Forks / Re: OXCE Lua

« on: November 10, 2024, 01:20:05 am »

Moved.

I worry that by being here, people will expect OXCE to get this at some point.

Yeah, I can completely understand that. Things have drastically changed since I first posted, so I'll edit the main post to reflect that this is a fork, and will remain a fork unless that decision changes in the future.

7

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:57:13 am »

I have no idea to be honest. It doesn't quite fit with "WIP", as that's for mods AFAIK. It doesn't quite fit in "Build and Ports" either. Perhaps just in the root "OpenXcom Extended" board? I just worry that by being here, people will either expect Brutal to get this at some point, or that the branch will support Brutal in the future, which we don't really have plans for, and I've already been told that Ximli wants to keep things simple and not pull in branches that contain such drastic changes as this one introduces(unless it's proven solid and there's enough support for it)

8

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:38:03 am »

Also, I'm not sure BOXCE is the best place for this project. I mean, it really has nothing to do with Brutal at all, other than I forked from there.

I mean, if OXCE is "A younger brother of OpenXcom focused on modding support and QoL." then this would be like the step son from a previous marriage after OXCE has grown up

9

More Forks / Re: BOXCE Lua

« on: November 10, 2024, 12:36:12 am »

I've posted an update:

https://github.com/ken-noland/OpenXcom/wiki/Update-%E2%80%90-November-9th-2024

In short...

What started as a simple Lua integration for OpenXcom turned into a major overhaul as each step uncovered more work. Moving to an Entity Component System (ECS) seemed like the best way to manage object lifecycles and interactions between Lua and C++, but it meant converting the game’s architecture piece by piece. This led to developing an entity inspector, which then revealed limitations with SDL’s event handling, pushing a switch to Vulkan. With SDL out of the picture, it made sense to revisit other areas—Options, Filesystem, Mods—that had grown cluttered over years of patching. Each fix led to another realization, eventually bringing about the need for a centralized Engine class to handle all the stuff SDL used to do, but outside of the Game class.

Switching to Vulkan and the ECS framework means resources are now managed through handles. With new Options class, a cleaner filesystem, and a CMake-based build system, the project is more cross-platform ready. The end goal is still to provide a more open foundation for modders, allowing Lua to prototype features that could later be optimized in C++. It has definitely taken longer than planned, but is still progressing.

I'm currently working on the oxce-lua-vulkan branch. I'll try to post updates more regularly now that the core is almost done.

10

Programming / Re: SDL2 branch status?

« on: November 09, 2024, 02:41:41 am »

I think it's fair to say that it is abandoned at this point. SDL1 is no longer offered on Mac and has been pulled from homebrew, and I didn't waste much time trying to find if I could download it elsewhere.

11

Brutal AI / Re: BOXCE for Windows XP

« on: November 09, 2024, 02:35:24 am »

wait, people still run the game on WinXP? Serious question, as I'm working on something that would definitely not work on XP.

12

OpenXcom Extended (OXCE) / Re: Converting OXCE to rapidyaml

« on: November 09, 2024, 02:29:11 am »

Hmmm... maybe worth knowing, but maybe not... I've switched a lot of the resource loading and unloading to Json for the Lua/Vulkan branch. Granted, it's not a released branch and I'm still a fair ways away from releasing it, but I thought it might be worth mentioning.

The way I deal with save game, mod loading(rulesets and so on) and config stuff is through serializers which are either hand crafted or auto-generated from the run time type information. What this means is that there is a considerable increase in compile times and executable size due in large part to the compilation and linking of a code generator which then needs further compilation, but saves time on loading during runtime. For Lua and Json, all the bindings are automatic, but since a goal of the Lua/Vulkan project is to retain backwards compatibility, then I manually put in the YAML serializers so they conform to the original OXCE rules. In fact, it makes it nicer because I can put whatever I want in the Json and not have to worry if I might break some mod out there that someone wrote a long time ago. Granted, I've only made it as far as the config stuff at the moment(config, localization, and a few other bits), so I might run into some issue down the line which might make it more of a problem, but working with Json in debug is so much faster than YAML. However, equally, it is worth mentioning that I barely notice YAML in release mode.

Use this information however you see fit. I'd be happy to see SOMETHING improve debug loading and unloading of mods. The Lua/Vulkan branch is really broken at the moment and I won't make any promises as to when it will be released, so there's also that to consider

13

More Forks / Re: OXCE Lua

« on: October 21, 2024, 01:21:29 pm »

Yet another update.

I decided to use run time type reflection to make the Lua binding easier, and in the process, I created the entity inspector. Early work on the entity inspector had it running in the same process as the game and using wxWidgets to manage the UI elements. Basically, anything that is an entity can have all it's components and their properties viewed and possibly edited by the inspector. However, I ran into a significant snag. It turns out that SDL hogs all the input for the process and there is no way for SDL and wxWidgets to live happily side by side in the same process. So, I decided to ditch SDL and rewrite the audio/video/input systems.

I've got it to the point now where most of the UI is rendering using Vulkan, and audio is through OpenAL, but that can be swapped out later if needed. I have a lot of work to do on the video playback, but most of the surface level stuff is functional.

I've ripped out the file handling and options logic and replaced it with a virtual filesystem and new options class because, well, to be honest, it was bugging me having things stored as global variables.

Everything I've done so far has been compiled on Windows and Linux. I don't have a Mac, although I did spend some time to see if I could spin up a Mac VMWare instance. That does appear to be working, but I haven't had a chance to get it all set up with homebrew and whatever else is needed. I did ditch all build systems other than CMake, so I'm not going to maintain the visual studio solution files or makefiles or any of that anymore. This is not a reflection on those build systems, but CMake can target almost every major IDE and handle the prerequisite gathering and all of that stuff without having to try to locate the right libraries for your system and place them in the right folder.

So, what I thought would be a few weeks of "just wrap things in Lua" turned out to be a major rewrite on all the core systems. I even changed up the log system so I could, in theory, have an in game console outside of the entity inspector. I've probably broken a few things along the way, but the general direction is good and seeing the front end menu pop up after weeks of uncertainty, well, really gave me a huge boost of morale.

14

More Forks / Re: OXCE Lua

« on: June 19, 2024, 03:03:58 am »

Update:

Right, there's been some big modifications to the original source. Maxmahem has graciously offered to help move towards a more Entity Component System compliant code base. The reason why is that we need handles to objects so we don't have to manage the object lifecycle as much on the Lua side. I've started working on the UI side, wrapping all the interface objects and converting those over to composable elements(components) to make things easier when translating to and from Lua.

Overall, this is shaping up to be a huge refactor.

I have a branch that I've been toying with where I take the Lua bindings and I redirect it to a json serializer. I open up a plain old TCP socket and I accept basic HTTP GET and POST commands which basically means I can now edit anything that is exposed to Lua through a web page. It's actually something I've done for a lot of projects I've worked on, and it greatly helps test and diagnose issues. I'll probably merge that in to the Lua branch soon, although it has very little to do with Lua.

Would love to get your thoughts on these changes. Feel free to leave feedback here, or on the repo, any time!

15

More Forks / OXCE Lua/Vulkan

« on: June 07, 2024, 03:23:35 pm »

Hey,

That's right, I've started it.

https://github.com/ken-noland/OpenXcom

I’m excited to introduce you to my ongoing OpenXcom fork, which started as a fork of Brutal (BOXCE) but has now evolved into a distinct project with its own vision and goals. While this project won’t merge with OpenXcom Extended (OXCE), the aim is to open new doors for modders by providing a foundation that’s as flexible and accessible as possible. You can learn more about OXCE here to see where this project diverges.

Project Goals

This project’s primary goal is to enable a level of scripting that hasn’t been possible before, using Lua—a scripting language widely recognized and commonly used in the modding community. Lua will allow modders to script almost everything in the game, from region behaviors and globe mechanics to detailed particle effects, providing an unprecedented level of customization. My hope is that Lua scripting will make OpenXcom modding accessible to a broader audience and allow for rapid prototyping and development without diving into the complexities of the underlying codebase.

As part of this overhaul, we are also upgrading the underlying systems to modern APIs by replacing SDL with Vulkan for graphics and OpenAL for audio. This update ensures that OpenXcom remains compatible with modern platforms and continues to be maintainable long-term.

Building Toward New Possibilities

With this new scripting system, there are some powerful modding possibilities within reach:

• Complete UI Overhauls: The scripting system allows modders to redefine the game’s UI, enabling complete overhauls if desired. Modders can customize surfaces, blocks, and interactive elements, creating tailored interfaces.
  
• Procedural Terrain Generation: Modders can define their own terrain elements and structures, potentially enabling procedural terrain for dynamic mission environments.
  
• Custom Attributes: Lua scripting supports defining custom attributes in any object, allowing you to add new stats or attributes without the need for complicated workarounds
  
• Dynamic Mission Types and Objectives: With scripting, modders could introduce custom mission types, adapting objectives based on player actions or randomized conditions.
  
• Custom AI Behaviors: Lua makes it possible to script custom AI tactics, from enemy behaviors to ally actions, offering more reactive and intelligent opponents.
  
• Environmental Effects and Weather Systems: Modders could add dynamic environmental conditions, such as weather or hazardous zones, that impact gameplay in real time.
  
• Campaign Progression and Branching Paths: The system could support dynamic campaigns with branching storylines, influenced by player choices and mission outcomes.
  
• Advanced Resource Management: Beyond custom attributes, modders could introduce complex resource systems, alternate economies, or even faction loyalty mechanics.
  
• Enhanced Mod Compatibility and Interactions: The Lua foundation allows for dynamic interaction between mods, creating new possibilities for layered, collaborative mod features.
  

Why Lua and Vulkan?

The choice to use Lua for scripting was driven by the desire to make OpenXcom modding as accessible and flexible as possible. Lua is a widely-used language with a low learning curve, making it easy for modders to jump in and create complex, game-changing modifications without needing to delve into C++ code. Its simplicity is paired with powerful functionality, allowing modders to define game behaviors, create custom attributes, and introduce unique gameplay mechanics. Lua also integrates smoothly with C++, making it an ideal choice for a project where scripts need to interact closely with the game’s underlying systems.

The switch from SDL to Vulkan was equally intentional. SDL has served the project well for years, but Vulkan offers greater control over graphics rendering and better performance on modern hardware. This control is critical in allowing OpenXcom to support advanced visual features and to run efficiently across platforms, including potential future extensions to Mac and Linux. Vulkan also supports multi-threading better than SDL, which is a huge advantage for performance as the game grows more complex. This shift paves the way for visual enhancements, optimized performance, and cross-platform support, ensuring OpenXcom is equipped to handle both the current goals and future expansions.

Runtime Type Generation and Serialization

A key feature in this project is the use of runtime type generation to handle serialization between the engine, JSON objects on disk, and Lua functions and tables. Using runtime type information, the system automatically generates code to serialize and deserialize data to and from the engine. This allows modders to define custom attributes directly in Lua, which can then be accessed within the engine or stored in JSON files for persistence. This approach reduces the need for extensive C++ modifications for each new attribute or entity, simplifying the integration of modder-defined data.

This setup relies on type reflection to dynamically retrieve type information and automate serialization code generation. Custom attributes or entities defined in Lua are accessible within the engine, with JSON handling save states and configurations without requiring a unique serializer for each type. By generating Lua and JSON serializers automatically, components—such as gameplay attributes, UI elements, or terrain features—are consistently accessible across scripting and storage.

To maintain backward compatibility with older mods, YAML serialization is handled through hand-coded serializers. This approach ensures that legacy mods developed with YAML remain functional without modification. The combination of runtime type generation, automatic Lua and JSON serialization, and manual YAML support provides a flexible system that accommodates both legacy and new modding capabilities.

Future Vision

Looking beyond the initial goals, this project opens up some ambitious possibilities for the future of OpenXcom modding. Here are a few of the ideas that could become feasible with the new foundation:

• Playing X-COM 1 and 2 in a Seamless Campaign: Imagine starting with soldiers from X-COM 1 and transitioning directly into X-COM 2, carrying over your personnel, resources, and accomplishments to fight new threats with the same team.
  
• Complex Geopolitical Dynamics: The new scripting system could allow for deeper political interactions, where countries and factions have evolving allegiances, conflicts, and responses to player actions. Modders could build complex systems of geopolitical intrigue, making each decision impact the broader world in meaningful ways.
  
• Fog of War: Enabling an ongoing fog of war across the global map would add a layer of mystery and tension to strategic decision-making, concealing alien bases, research installations, and potentially hostile factions until they’re scouted or discovered.
  
• Multiple Globes for Interplanetary Operations: This new foundation makes it feasible to support multiple globe views, allowing interplanetary missions where players can explore, engage, and defend across different planets, each with its own conditions and alien threats. An interplanetary scope would expand the tactical and strategic gameplay far beyond Earth.
  

With these possibilities, OpenXcom could evolve into a platform that accommodates far more complex, immersive, and expansive gameplay. For anyone interested in following or contributing to this vision, I’ve outlined the project roadmap on the project’s GitHub Wiki. While we’re not yet at a public release stage, nightly builds are planned as we make further progress.

Thank you to everyone following and supporting the project. This is still a work in progress, but I’m excited to see what new experiences this foundation will enable in the OpenXcom universe. Stay tuned for more updates as we continue expanding OpenXcom’s possibilities!