[OXCE+] Lua Integration

[Whispers]

but Lua shroud obey game objects life cycle, if some object right now is created by function A then Lua should use this function too to create object. Same with deleting. If something is not changeable by game engine scripts usually should not change this too, this is important in case when some game logic really on that value is not changed.

Best if you could control per script call, some scripts could be allowed to change unit in any way other can only read.

Hmm, not really easily done with the way I have started implementing the API. I do have a container class I quickly wrote which maps user objects to their respective class pointers, so maybe in that container class I could write some access rights management, but there will be HUNDREDS of hooks. On<Whatever Event I Put A Hook For> would then set the access rights to object classes, but that seams extreme. Can you point to a situation where one thing can be read, but not written during a single event, yet can be read/written on other events?

You can compare it with current scripts, if your changes will be deleting my function parameter and replacing by your then its 100% fine. But if you need change every game logic class then is not fine.

The hooks will need to be implemented in the logic classes so I do imagine it touching most logic classes

Whole point is that I'm not interesting in maintaining two script engines in OXCE (and probably Meridian too) this is way I set high-bar that new script engine is objective better in every aspect compare to my engine. Otherwise I will stick to my creation This mean you will need do more work but I would be unwise to give up couple of years of work for free :>

I get that. I'm actually saying I will maintain the Lua stuff and you could continue to maintain the existing script stuff.

Instead of wiki I would be more interested in WiP commits that show how it will work.

Problem is that I'm not allowed to touch a source code editor over the weekends. My wife is very strict about that

Writing the wiki also helps me clear my thoughts. It helps make sure what I'm working on will actually work.

I'll push up something later today to show progress.

Figured I'd throw my two cents in, because I've always been curious about integrating Lua scripting (I've played with it in other games), but have no idea how. Usually when people talk about scripting they think about modders doing anything, limitless possibilities! I don't think that's the point. OpenXcom is already open-source, the possibilities are already limitless, so clearly it's not that simple. At the end of the day, modders are modders and programmers are programmers and I don't think scripting is what will convert them. But in my eyes it has the potential to help solve one of OpenXcom's biggest problems: maintainability. Now that vanilla is basically done, all that's left is adding modding features, forever and ever...

Everytime a mod needs a feature, a modder has to convince a programmer to add that feature, and that feature is then available to every single mod. This poses multiple problems. Every feature has to be compatible with every other feature, and as mods grow, so will the engine. Features are never removed, only added, and the weight is on every programmer that follows. Every feature, no matter how specific and particular, has to be generalized to work for everyone. Etc. Does it make sense for the weight of every mod to be on every other mod? On every programmer? Probably not. It's a maintainability and scalability nightmare, and in the end programmers have to be choosy and decide what is worth supporting or not, fragmenting the modders and codebases.

So, instead of the engine having to support every feature for every mod, what if mods could carry their own code? Their own features? Instead of Items needing 100+ properties, what if they could instead have scriptable functions for onFire, onUse, onHit, etc. This would both give mods more power and make code lighter and easier to maintain for programmers. Now I don't think this means exposing everything and creating a completely generic scriptable engine, that's its own nightmare. Just exposing the most common "game events" from which modders can manipulate the simulation would suffice, letting them add their own associated state and UI if needed. A happy middleground between modder potential and programmer scope.

So why Lua vs Yankes scripts? Well it's pretty clear adoption has not been great, modders would rather bug programmers than deal with it, and the only programmers that deal with it are... Yankes. And I think a big factor is because modders aren't programmers. You can give them all the code and potential in the world, if they don't know how to use it, it's useless. So will they code Lua? Probably not. The draw of scripting isn't to the modders who can't code, but to the programmers who can't code C++ but can deal with something simpler like Lua, which in turn help the modders. I doubt they're itching to learn a whole new bespoke scripting language.

As for performance... honestly, you could do a lot worse than Lua. It's the most common lightweight scripting language around, designed for embedded, runs on every platform on the planet. It'll never beat native, it's a bytecode VM, but for anything outside a tight game loop it should be fine. After all, OpenXcom struggles on anything under a 2Ghz and does all its graphical operations on the CPU, so I don't think we're winning any awards for performance either.

Yup

[Whispers]

Hmm, trying to think of a way to do terrain generated pressure plates. Something that is only executed once a soldier is standing on it. That could be another mod that I write to showcase the API.

[Whispers]

Updated OP with more updates

[Yankes]

So why Lua vs Yankes scripts? Well it's pretty clear adoption has not been great, modders would rather bug programmers than deal with it, and the only programmers that deal with it are... Yankes. And I think a big factor is because modders aren't programmers. You can give them all the code and potential in the world, if they don't know how to use it, it's useless. So will they code Lua? Probably not. The draw of scripting isn't to the modders who can't code, but to the programmers who can't code C++ but can deal with something simpler like Lua, which in turn help the modders. I doubt they're itching to learn a whole new bespoke scripting language.

And ohartenstein23 Even if I myself would use this scripts it still would be useful as way to write features that I do not like or do not wan't to hardcode.
I'm aware that my script engine is not perfect but adding another script engine on top of this will not fix it but even worse that.
This is why I agree to include Lua but only when it completely replace my engine without loose of most of functionality.

As for performance... honestly, you could do a lot worse than Lua. It's the most common lightweight scripting language around, designed for embedded, runs on every platform on the planet. It'll never beat native, it's a bytecode VM, but for anything outside a tight game loop it should be fine. After all, OpenXcom struggles on anything under a 2Ghz and does all its graphical operations on the CPU, so I don't think we're winning any awards for performance either.

This depend on usage, in most cases this not matter but if it allocate memory per pixel drawn? This is one of my script usages and some mods use this.
I can only guess how Lua will behave and what will be performance. If Whispers will show it work fine, then is not problem to drop my scripts but if otherwise it will be problematic.
My whole point is that can be only one script engine in code base (Lua or my) other wise will be one big maintaining mess.

Hmm, not really easily done with the way I have started implementing the API. I do have a container class I quickly wrote which maps user objects to their respective class pointers, so maybe in that container class I could write some access rights management, but there will be HUNDREDS of hooks. On<Whatever Event I Put A Hook For> would then set the access rights to object classes, but that seams extreme. Can you point to a situation where one thing can be read, but not written during a single event, yet can be read/written on other events?

only read when you draw unit or calculate fire reaction, both should not change any thing because it could break code.
on event wen unit is hit you can alter unit (kill it, change damage values, change some custom property).

Another important thing is random seed, when you would access it on unit draw this would make that bugs become ultra hard to debug.
This why in my scripts access to random is only allowed in moments when other data is changed.

Overall it could be simplify to two global states, one is allowed to modify thins other is not.

The hooks will need to be implemented in the logic classes so I do imagine it touching most logic classes

For number of changes/modifications to source code, look how I manage my scripts. If your solution is comparable then is fine.
In my case most "mess" is in save game objects `.cpp` files where I define all bindings and scripts api:
https://github.com/Yankes/OpenXcom/blob/OpenXcomExtended/src/Savegame/BattleUnit.cpp#L3882

And "links" visible publicly:
https://github.com/Yankes/OpenXcom/blob/OpenXcomExtended/src/Savegame/BattleUnit.h#L148
or
https://github.com/Yankes/OpenXcom/blob/OpenXcomExtended/src/Mod/RuleItem.h#L183


If your changes are similarly localized and do not leak every other place then is fine.

I get that. I'm actually saying I will maintain the Lua stuff and you could continue to maintain the existing script stuff.

And after year you get bored and stop updating it or opposite I get bored and stop contributing to OXCE. In both cases will stay big blob of code that nobody want to touch. Right now I not interested in changing script engine or maintain another one but if you fulify my condition then I will accept it and take responsibility of maintaining it further.

[Whispers]

Overall it could be simplify to two global states, one is allowed to modify thins other is not.

Unfortunately it's not that easy. Once a Lua context is created, the API is injected into the context. Once the API is injected, I can't easily turn functions on or off depending on the context that they are run in, unless I block out functions based on the calling event.

If your changes are similarly localized and do not leak every other place then is fine.

Yeah, I go one step further. I have to insert source in events in the logic object that, well, triggers the event, but otherwise all functions and script bindings are done in completely separate source files and only access public member functions of existing objects. There's never any concern about "leakage" since I never allocate anything.

I did have to tweak how mods are loaded in order to get the path of the mod that is trying to execute the script, but otherwise it's just a matter of putting in hooks and creating get/set functions

And after year you get bored and stop updating it or opposite I get bored and stop contributing to OXCE. In both cases will stay big blob of code that nobody want to touch. Right now I not interested in changing script engine or maintain another one but if you fulify my condition then I will accept it and take responsibility of maintaining it further.

Heh, not likely to happen, but I get your point. Being honest, I'm a bit disappointed in myself for not finding OpenXcom earlier. I could have done this years ago

[Yankes]

Unfortunately it's not that easy. Once a Lua context is created, the API is injected into the context. Once the API is injected, I can't easily turn functions on or off depending on the context that they are run in, unless I block out functions based on the calling event.

One solution is create two contexts, if Lua job is only glue code then it will work fine, but if you want put lot of custom state in it then it will not work.
Another is add two different bindings for same type, it will be `BattleUnit` and `BattleUnitConst`, depending on details it could end up as big hack.

Yeah, I go one step further. I have to insert source in events in the logic object that, well, triggers the event, but otherwise all functions and script bindings are done in completely separate source files and only access public member functions of existing objects. There's never any concern about "leakage" since I never allocate anything.

"leak" not like from "memory leak" but like in "abstraction leak".

I did have to tweak how mods are loaded in order to get the path of the mod that is trying to execute the script, but otherwise it's just a matter of putting in hooks and creating get/set functions

All refactors are welcomed, if you refactor half of code base and it simplify and improve code quality and then it fine, I only do not want brutal force solution that make code ugly (like function with 20 arguments, 100 copies of same code, e.t.c.). Change of `FileModInfo`  is good example because I can throw away Lua changes and still include it to OXCE.

[Whispers]

One solution is create two contexts, if Lua job is only glue code then it will work fine, but if you want put lot of custom state in it then it will not work.

Contexts store all the globals for a lua script, and I imagine there will be a lot of people who want to access their globals of the ordinary script execution.

Instead, this means the concept of using the registerHook is more applicable. When the code executes a registered hook, it simply sets a flag in ModLuaScript as to what is calling the hook, then when the script calls into a C function, its up to the bound C function to get the current ModLuaScript to check the calling context and can then selectively throw an error. I'm not a fan of doing that, but it fills the requirement.

Another is add two different bindings for same type, it will be `BattleUnit` and `BattleUnitConst`, depending on details it could end up as big hack.

Eeeeks... My eyes, my eyes!

All refactors are welcomed, if you refactor half of code base and it simplify and improve code quality and then it fine, I only do not want brutal force solution that make code ugly (like function with 20 arguments, 100 copies of same code, e.t.c.). Change of `FileModInfo`  is good example because I can throw away Lua changes and still include it to OXCE.

Heh, so you're snooping on my changes already

Yeah, not a fan of using std::pair the way it was being used, so I threw that in there quickly because I needed to get the path to the mod, and refactoring felt better than hacking in another std::pair

[Yankes]

Yeah, not a fan of using std::pair the way it was being used, so I threw that in there quickly because I needed to get the path to the mod, and refactoring felt better than hacking in another std::pair

`std::tuple` for rescue ;P

But for more serious answer, pair or tuple are good as implementation detail, not as public interface. Adding name give greater meaning to code that some random pack of values.

[Whispers]

`std::tuple` for rescue ;P

Where's my hammer? I've got to knock some sense into...

But for more serious answer, pair or tuple are good as implementation detail, not as public interface. Adding name give greater meaning to code that some random pack of values.

Oh, okay, I'll put my hammer away now.

[Whispers]

Quick question to @Yankes and @Meridian.

What are your thoughts on std::shared_ptr and std::unique_ptr, etc?

Reason I ask is because it would make a large portion of the Lua code a lot easier. Instead of having to check for invalid inputs, I can guarantee a pointer will never be null simply by increasing a reference count.

It's not necessary, but thought I would ask.

[Yankes]

for me ++++++++++++i;

I my self use `std::unique_ptr` in in my recent refactor of surfaces to enable `std::move`.

[Whispers]

Ugh, I'm not really happy with the "register callbacks" mechanism. It lacks a certain grace that I usually like my APIs to have, but I simply can't think of a better way to deal with it.

Scenario:

As a mod developer, I want a show, every day in game time, my current funds

Solution:

Using the register callback mechanism, you can specify a lua function to be called. Something like this:

Code: [Select]

function on1Hour(gametime)
  print("Your current funds are " .. tostring(xcom.game.getFunds()))
end

xcom.geoscape.registerOn1HourCallback(on1Hour)

Now, every hour, on the hour, the function will be called. If you try to register more than one function, then it will simply overwrite whatever callback you had previously. Perhaps I should return the previous function? Maybe set it up so there is a has<x>Callback and clear<X>Callback? Hmmm, I'm not sure. I need to get this right because the callbacks are the key to the whole API.

[ivandogovich]

Scenario:

As a mod developer, I want a show, every day in game time, my current funds
<snip>

Or just go with the option built into the game engine currently.




and Yeah, I know you were just giving an example, just pointing out that in this instance, its already there.

[Whispers]

and Yeah, I know you were just giving an example, just pointing out that in this instance, its already there.

The point was not to showcase new features, but how I go about building the API functions.

[Yankes]

In my version I solve this as:
- You only add callbacks (no remove)
- Each callback have floating point "index"
- Callbacks are call in order of index

After load index can be drop as only relative order is important. Float vales are for finding unique position between two other events.
Another thing is that in my version I have global and local scripts. Local mean that it belong to some game object and global are shared between all objects of same type. At index `0.00` local script is called that is unique for each object, if some global script have `-1` index it will be call before local, for `1` it will be call after.