aliens

Author Topic: [OXCE+] Lua Integration  (Read 25869 times)

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
[OXCE+] Lua Integration
« on: September 15, 2018, 04:55:31 pm »
I started work on extending OXCE+ to include LUA hooks and so far the results are promising.

I wanted to start this thread to discuss the progress and also address any design considerations with how the OXCE+ LUA API is constructed.

First up, the progress.

I've set up a new rule which allows you to specify a script to load. The rule syntax is fairly straightforward.

Code: [Select]
game:
  script: scripts/game.lua

This will simply tell the engine that you have a script in <mod root>/scripts/game.lua. When a game is loaded or a new game is started, it then executes the script for the first time. When the script is executed, it is responsible for registering the hooks that it requires in the engine. Currently, it's implemented in such a way that you can only have one hook per script per callback, so you can call xcom.geoscape.register1HourCallback(function) multiple times, but it will only execute the last function you pass to it on every hour.

In the current version on github, there is one callback at the moment. I'll be refactoring the code a bit to make it more generic so other callbacks will be quicker to implement, however, start small and work your way up :D

With that working, the next steps are to add in a ton of global callbacks and hooks.

I set up the github repo. The repo with the Lua additions is here:

https://github.com/xxWhispers/OpenXcom

Why Lua? Well, to be honest, it's a widely known language that I have familiarity with implementing and developing with. That's literally the only reason why. Oh, and functions are data, which allows you to overwrite other functions to extend functionality in ways that are limited only to your imagination(and coding skills).

From a design perspective, I want to be clear that the idea is not to replace or port any of the existing code over to Lua. I don't want to port OXCE+ over to Lua. Instead, the goal is merely to provide the ability to get/set data internally without having to modify the engine code for new features.

With that said, there are things I do have in mind to allow you to extend certain features already built into the engine. For instance, one of my goals is to allow you to write your own terrain generation algorithms and hook into the existing terrain generation to allow you to overwrite or skip certain terrain passes. Another thing I want to allow you to do is customise almost every screen in the game by exposing the active UI elements and allow you to move/add/remove all UI bits around as you see fit. Then there's the obvious stuff, like being able to change soldier stats, add new fields, and completely new game mechanics(soldier mana stat anyone?)

Future plans:

* Have the ability to set/get existing base information such as stores, tiles, crafts, soldiers, etc.
* Add more triggers like onEnemyCraftDetected, onEnemyLanded, onEnemyBaseDiscovered, etc
* Have custom terrain generation scripts
* Have battlescape scripted events
* Hook into save/load game to allow mods to save their stuff.

Questions for modders:

* Should there be scoped/privileged Lua calls that are only accessible to certain mods?
* Any special requests for features? ( I have a feeling I may regret asking this :P )

So I'd really like to get your thoughts on this. Any questions, please feel free to give me a shout! I will be back here to post new updates as they come, so stay tuned!
« Last Edit: September 19, 2018, 07:08:35 pm by Whispers »

Offline bulletdesigner

  • Commander
  • *****
  • Posts: 685
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #1 on: September 15, 2018, 05:09:50 pm »
Sound promising 8)

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #2 on: September 15, 2018, 08:04:23 pm »
Updated the OP... more updates to come

Offline Yankes

  • Global Moderator
  • Commander
  • ***
  • Posts: 3347
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #3 on: September 15, 2018, 08:30:56 pm »
There are already scripts in OXCE, right now only available in battlescape but can be expanded to geoscape too.
And now adding new scripts that are incompatible with old ones?
Of corse my scripts are right now limited (not even Turing complete right now) but are closely coupled to C++ and support `const`.
Another thing is that limitation is good thing that I can control what is possible in each script and what it have access to.

For me this make hard do justify dropping my scripts in favor of LUA especially that I doubt it will be capable of fulfil all current usage.
Example of that is how much overhead have running LUA script? How many thousand of times can be call per frame?
What you need to change in source to allow this integration, and what dependency will be needed to compile with LUA enabled?

Of corse I can't stop doing this as this is open source project and you can do any thing you want with code.
Simply I not convinced to adding it to main branch of OXCE or OXCE+.

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #4 on: September 15, 2018, 08:50:16 pm »
Hi Yankes!

Yes, I heard all about the yankesScript and I've briefly browsed through what you have there. Yes, you've done a really great job at making a logic graph, and I'm not trying to replace yours for the thousands of scripts that already exist. However, I do think there is a need for something more accessible to the modding community, and Lua fits that description. Instead of having to learn about the quirks of one language in order to make something unique, you can learn a language used by thousands of other games(or re-use your existing knowledge of Lua, if you are already familiar with it).

I can see that you are very much someone who cares about performance. I used to be that way also. LuaJit exists to fill that gap by precompiling the Lua on the target platform, if performance ever becomes an issue, which I doubt it ever will be. For now, I don't think it will be an issue. Back in 2006, I used Lua on Supreme Commander, and we had thousands of units running multiple scripts every frame, and that was 10 years ago.

Const correctness is an interesting topic, and one that even C++ developers often get wrong. Did you know that the C/C++ compilers(GCC, Clang, and MSVC) completely ignore const when it comes to optimisations(use preprocessor definitions if you want a true const)? Reason why is that you can easily un-const something without the compiler ever knowing. All it is is simply a hint to the programmer that they shouldn't modify the value. Anyway, I digress. I don't want to start a C/C++ standard discussion because those could go on for days. I do, however, want to discuss Lua :D

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #5 on: September 15, 2018, 08:53:33 pm »
Oh, and the only dependency is Lua5.1. It consists of a handful of source files and can be retrieved and compiled on any platforms with one bash call. You're using Boost in OXCE and OXCE+, so I hardly think the 1-2k size would make a difference.

Offline Yankes

  • Global Moderator
  • Commander
  • ***
  • Posts: 3347
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #6 on: September 15, 2018, 09:31:27 pm »
Hi Yankes!

Yes, I heard all about the yankesScript and I've briefly browsed through what you have there. Yes, you've done a really great job at making a logic graph, and I'm not trying to replace yours for the thousands of scripts that already exist. However, I do think there is a need for something more accessible to the modding community, and Lua fits that description. Instead of having to learn about the quirks of one language in order to make something unique, you can learn a language used by thousands of other games(or re-use your existing knowledge of Lua, if you are already familiar with it).
Graphic was only start point of developing this scripts, right now they expand to other places. My concerns is that each script engine is burden on source code (for me my script is lesser burden but for other programmers this could be opposite) to maintain to incompatible engines.

I can see that you are very much someone who cares about performance. I used to be that way also. LuaJit exists to fill that gap by precompiling the Lua on the target platform, if performance ever becomes an issue, which I doubt it ever will be. For now, I don't think it will be an issue. Back in 2006, I used Lua on Supreme Commander, and we had thousands of units running multiple scripts every frame, and that was 10 years ago.
Speed of my script engine rely on it simplicity (implementation is complex but it all boils down to switch in loop) and using same memory model as C++. LUA have different model and need some conversions to transfer data (I could be wrong there, best if you would show code in action).
Another thing is LuaJit have cost too (on start up) and probably is big as whole OXC exe (again I did not see your implementation to be 100% sure).
In some UI parts it not matter but it should replace my script that need this.

Const correctness is an interesting topic, and one that even C++ developers often get wrong. Did you know that the C/C++ compilers(GCC, Clang, and MSVC) completely ignore const when it comes to optimisations(use preprocessor definitions if you want a true const)? Reason why is that you can easily un-const something without the compiler ever knowing. All it is is simply a hint to the programmer that they shouldn't modify the value. Anyway, I digress. I don't want to start a C/C++ standard discussion because those could go on for days. I do, however, want to discuss Lua :D
You completely misunderstood meaning of `const` especially in context of my script (where I do not allow removing it). This allow me send some object to script and do not worry that someone modify it without my permission. This is great tool to reason on what happens in code and script that are called.


Oh, and the only dependency is Lua5.1. It consists of a handful of source files and can be retrieved and compiled on any platforms with one bash call. You're using Boost in OXCE and OXCE+, so I hardly think the 1-2k size would make a difference.
OXCE do not use boost. This was yaml dependency that was drop by them when they switch to C++11. And more importantly it will work with VS? this is windows that do not have bash. Another important toolchain to consider is mxe (gcc based but have predefined libs available). This is why I ask ablaut dependencies.

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #7 on: September 15, 2018, 10:37:23 pm »
My concerns is that each script engine is burden on source code (for me my script is lesser burden but for other programmers this could be opposite) to maintain to incompatible engines.

Ah, okay. Well, having modders make demands/hacks into code I see as a larger burden, and one that leads to more issues as it progresses. Having a script API that you agree upon at the start and then slowly adding to it is more flexible, IMO.

Speed of my script engine rely on it simplicity (implementation is complex but it all boils down to switch in loop) and using same memory model as C++. LUA have different model and need some conversions to transfer data (I could be wrong there, best if you would show code in action).

If you implemented Lua using default stack operations only, then yes, you have to convert values to/from script. However, there are user variables that you can define which use the memory address you give it and type you specify, however, I wouldn't go that route. Lua is FAST. I mean, seriously fast. I suppose we could do a side by side comparison using some algorithm(s), like md5, or perhaps something more memory intensive like huffman encoding, but I totally see your point about it being too slow for graphics purposes. For that, you would need something a little closer to the metal.

For comparison, the setting of 1 variable actually sets 3 points in contiguous memory. The first point is the ref counter, the second is the data type, and the third is the data itself. There isn't a single conditional in that code, which  means that it runs faster then your average if statement. Getting a value has one conditional, which is merely a conditional to ensure the value in memory is in the correct format - otherwise convert it, or throw an exception the old school longjmp way, not the fancy and expensve stack unwinding C++ way. Since the logic is run by token static single assignment, it's just as fast, and in a lot of cases faster than a logic tree switch statement.

I didn't build Lua, but if I were to build a scripting language, I would do something very similar to what they have done.

Another thing is LuaJit have cost too (on start up) and probably is big as whole OXC exe (again I did not see your implementation to be 100% sure).
In some UI parts it not matter but it should replace my script that need this.

Yes, and that is one of the biggest drawbacks of LuaJit. I'm not a fan of using it unless we do run into performance issues at runtime. I don't see that being an initial problem though, unless someone writes a script for every tile tha tgoes through every other tile of the level and every pixel at the same time. (I could totally see someone attempting to do that). In those cases, there's Lua's interrupt ability, which allows you to pause currently executing scripts and report errors/warnings to developers basically telling them to stop hogging all the resources!

You completely misunderstood meaning of `const` especially in context of my script (where I do not allow removing it). This allow me send some object to script and do not worry that someone modify it without my permission. This is great tool to reason on what happens in code and script that are called.

Oh, that's handy! Out of curiosity, instead of creating a whole system that then needs to be guarded by conditional if(s) which cause pipeline execution slowdowns(branch prediction is the largest CPU performance killer after cache misses because it causes pipeline flushes)... why not just copy it and let them modify the copy of the const so it doesn't have any effect on the original const variable?

OXCE do not use boost. This was yaml dependency that was drop by them when they switch to C++11.

Yeah, I've never used YAML in any projects before. This is my first time playing with it and I've already got mixed feelings about it. I did, however, quickly pull the latest luaYaml to allow me to get the config stuff exposed to Lua. So far, no problems there :D

Also, do you know what version they switched to C++11? I'm running 0.5.1 at the moment, but have considered updating that.

And more importantly it will work with VS? this is windows that do not have bash. Another important toolchain to consider is mxe (gcc based but have predefined libs available). This is why I ask ablaut dependencies.

Ah, yes, of course Lua works on MSVC. For MXE, I may need a bit of help on that, but I don't see that as a major issue.

I suppose the question I have is what would it take to get you to accept this? I'm not going to start up another branch just because of adding in Lua. That seams excessive and would further fragment the modding community. My biggest reason for doing this is because I wanted to do a quick mod and someone told me I'd have to muck about the engine code and release my own executable version if I wanted to do it, but that doesn't feel like a mod anymore. Simply exposing a bunch of stuff from the engine to a scripting interface is sufficient for what I wanted to do, so I started looking in to it. If you're saying you won't take it, then there's no point in doing it.

Offline Yankes

  • Global Moderator
  • Commander
  • ***
  • Posts: 3347
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #8 on: September 16, 2018, 03:04:21 am »
Ah, okay. Well, having modders make demands/hacks into code I see as a larger burden, and one that leads to more issues as it progresses. Having a script API that you agree upon at the start and then slowly adding to it is more flexible, IMO.
We compare not "script<->not scripts" but "2 scripts engine <-> 1 script engine". I add script exactly because of this reason you mentioned, but this can't be justification for another engine if one exists.

Another thing is demand for scripts in OXCE, right now I only one who push for new scripts (as my pet feature). Current interest is moderate (could be caused by limited scope of my scripts).

If you implemented Lua using default stack operations only, then yes, you have to convert values to/from script. However, there are user variables that you can define which use the memory address you give it and type you specify, however, I wouldn't go that route. Lua is FAST. I mean, seriously fast. I suppose we could do a side by side comparison using some algorithm(s), like md5, or perhaps something more memory intensive like huffman encoding, but I totally see your point about it being too slow for graphics purposes. For that, you would need something a little closer to the metal.
Probably best benchmark for me would be custom bliting (altering pixel values on drawing surfaces).
Simply replace `ScriptWorkerBlit::executeBlit` with LUA call that will do same work without any slowdown.
Some thing like this: https://www.youtube.com/watch?v=FKzqCOg2DrQ (only units graphic, around 24 at once on screen).

For comparison, the setting of 1 variable actually sets 3 points in contiguous memory. The first point is the ref counter, the second is the data type, and the third is the data itself. There isn't a single conditional in that code, which  means that it runs faster then your average if statement. Getting a value has one conditional, which is merely a conditional to ensure the value in memory is in the correct format - otherwise convert it, or throw an exception the old school longjmp way, not the fancy and expensve stack unwinding C++ way. Since the logic is run by token static single assignment, it's just as fast, and in a lot of cases faster than a logic tree switch statement.

I didn't build Lua, but if I were to build a scripting language, I would do something very similar to what they have done.
And there is big difference, because my scripts do not allocate any memory only get buffer from C++ stack where they place variables. I simply traded speed for some limitations. If I would need allocate something then I would ask game code to create if for me.


Yes, and that is one of the biggest drawbacks of LuaJit. I'm not a fan of using it unless we do run into performance issues at runtime. I don't see that being an initial problem though, unless someone writes a script for every tile tha tgoes through every other tile of the level and every pixel at the same time. (I could totally see someone attempting to do that). In those cases, there's Lua's interrupt ability, which allows you to pause currently executing scripts and report errors/warnings to developers basically telling them to stop hogging all the resources!

Oh, that's handy! Out of curiosity, instead of creating a whole system that then needs to be guarded by conditional if(s) which cause pipeline execution slowdowns(branch prediction is the largest CPU performance killer after cache misses because it causes pipeline flushes)... why not just copy it and let them modify the copy of the const so it doesn't have any effect on the original const variable?

This object that script edit is pointer to `BattleUnit`. you can't have copy of unit because this will miss point of using game objects.
Another thing my scripts are static typed. This mean if you try modify const object then script will refuse to parse.

Yeah, I've never used YAML in any projects before. This is my first time playing with it and I've already got mixed feelings about it. I did, however, quickly pull the latest luaYaml to allow me to get the config stuff exposed to Lua. So far, no problems there :D

Also, do you know what version they switched to C++11? I'm running 0.5.1 at the moment, but have considered updating that.
If this is from current github repo then its need C++11, if it from some else it probably still use C++98 and boost.

Ah, yes, of course Lua works on MSVC. For MXE, I may need a bit of help on that, but I don't see that as a major issue.

I suppose the question I have is what would it take to get you to accept this? I'm not going to start up another branch just because of adding in Lua. That seams excessive and would further fragment the modding community. My biggest reason for doing this is because I wanted to do a quick mod and someone told me I'd have to muck about the engine code and release my own executable version if I wanted to do it, but that doesn't feel like a mod anymore. Simply exposing a bunch of stuff from the engine to a scripting interface is sufficient for what I wanted to do, so I started looking in to it. If you're saying you won't take it, then there's no point in doing it.
a) Have better capabilities that my scripts (easy to do).
b) It do not have big overhead (probably harder).
c) Can replace all current usage of my scripts.
d) Do not disturb code base to much (e.g. you do not change multiple of functions signatures to pass some state).
e) If LUA have state it should be able to restore it after game save and load.
f)  Meridian is fine too to change script engine.
g) mxe work without any hacks
h) Have "killer app" for people like Dioxine,  Solarius Scorch, Hobbes or other big modders
With all this I will happy kill my scripts and replace it with LUA

btw If your mod feature align with my or Meridian goals then is possible to include it to OXCE+ directly without need for script engine.

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #9 on: September 16, 2018, 01:59:14 pm »
I had a longer post typed up, but due to power loss I guess I'm being forced to be brief :D

We compare not "script<->not scripts" but "2 scripts engine <-> 1 script engine". I add script exactly because of this reason you mentioned, but this can't be justification for another engine if one exists.

One of the biggest design tenants is that I do not want to replace or port any existing functionality so I do see yankesScript continuing for now. I'll maintain and expand the Lua API, and you can continue to focus on expanding yankesScript if you would like.

Another thing is demand for scripts in OXCE, right now I only one who push for new scripts (as my pet feature). Current interest is moderate (could be caused by limited scope of my scripts).

There's a huge demand from the modders I've spoken to on Discord.

Probably best benchmark for me would be custom bliting (altering pixel values on drawing surfaces).
Simply replace `ScriptWorkerBlit::executeBlit` with LUA call that will do same work without any slowdown.
Some thing like this: https://www.youtube.com/watch?v=FKzqCOg2DrQ (only units graphic, around 24 at once on screen).

Sounds like a good benchmark! I'll sneak that in as a "psychedelic" mod :D

And there is big difference, because my scripts do not allocate any memory only get buffer from C++ stack where they place variables. I simply traded speed for some limitations. If I would need allocate something then I would ask game code to create if for me.

The Lua context(which holds the memory for the Lua scripts) will initialise only with function pointers to allow the user to CRUD(create, read, update, delete) game objects. This means we can limited things to just "read only" if we want, but also means that no game state information will be held in the Lua context itself.

This object that script edit is pointer to `BattleUnit`. you can't have copy of unit because this will miss point of using game objects.
Another thing my scripts are static typed. This mean if you try modify const object then script will refuse to parse.

As mentioned above, we can limit access to read only since nothing will be stored in the Lua context.

If this is from current github repo then its need C++11, if it from some else it probably still use C++98 and boost.

Thanks! I'll pull the latest and compile from there instead of using Boost(that should cut compile time down a bit)

a) Have better capabilities that my scripts (easy to do).

Done :D

b) It do not have big overhead (probably harder).

Lua doesn't have a big overhead.

c) Can replace all current usage of my scripts.

Why is this a requirement? I'm not trying to replace yankesScript, at least, not initially. I'm only trying to provide a mod(der) friendly interface for other people to make what they want.

d) Do not disturb code base to much (e.g. you do not change multiple of functions signatures to pass some state).

Might need to define "too much" :D

I'll have to change the interface for loading/saving objects(almost everything in SaveGame, and most things that already take YAML node as a parameter) to include the lua context so that I can pass data to/from the game state.

e) If LUA have state it should be able to restore it after game save and load.

Yeah, still thinking on the best way to handle persistence. My current thoughts are to create a yaml node for game objects that stores the data per each mod, but some saves are already getting too large, so I might need to think about this some more.

f)  Meridian is fine too to change script engine.

Cool!

g) mxe work without any hacks

I don't think this'll be an issue

h) Have "killer app" for people like Dioxine,  Solarius Scorch, Hobbes or other big modders

"Hello World" isn't enough :P

I'm compiling a list of "example mods" as a way of specifying a Minimal Viable Product(MVP) milestone. So far, I have the following:

* add new field to the soldier for mana, and show how that can be used to implement new gameplay mechanics
* modify world zones and restrict missions in the zones to allow for diplomacy as a new mechanic
* implement "traitors" which are enemies that flip sides during battle and join your team
* open for more suggestions!

With all this I will happy kill my scripts and replace it with LUA

Like I said, the goal is not to kill yankesScript, but to supplement it initially. If there comes a point where the two collide, then yeah, we may want to revisit this strategy, but otherwise I don't see a problem with them co-existing :D

btw If your mod feature align with my or Meridian goals then is possible to include it to OXCE+ directly without need for script engine.

Yeah, but that simply continue the current process for writing custom logic in mods, which is to modify the engine in order to make something new. I would like to see what else the modders come up with given nearly full access to the internals of the engine :D

Online Meridian

  • Global Moderator
  • Commander
  • ***
  • Posts: 9078
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #10 on: September 16, 2018, 02:39:38 pm »
I would like to see what else the modders come up with given nearly full access to the internals of the engine :D

FWIW, I would like to see what "nearly full access to the internals of the engine" actually means... because even after reading all of this, I still have no idea.
For me, right now, this is something I can't even imagine, let alone implement.
Or maybe I just imagine too much and it means something a lot less full-access-ey.

Also, please be careful with such words... our modders are not developers (with a few exceptions)... and they will believe you when you say they will be able to do "anything they want"... which is just unfair to them. E.g. I'm pretty sure Hobbes will not be able to "script" an online multi-player support, which he wants so much.

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #11 on: September 16, 2018, 02:54:31 pm »
Ah, yes.. forgot my audience for a minute there :D

By "full access" I mean basically anything you can see in the save file, and also what is loaded in the rules, with some exceptions to anything that may totally break the game(for example, you are allowed to see what mods are loaded, but you can't add new mods to that list on demand)

Cheers,
-Ken

Offline Whispers

  • Sergeant
  • **
  • Posts: 43
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #12 on: September 16, 2018, 03:29:37 pm »
For lack of a better place to put it, I'm going to start putting the lua API documentation in github.

https://github.com/xxWhispers/OpenXcom/wiki

Right now I've only just started putting together the list of available functions, but I suppose I should start somewhere :D

I'll add a "getting started using lua" once the project is in more of a stable state.

Offline Yankes

  • Global Moderator
  • Commander
  • ***
  • Posts: 3347
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #13 on: September 17, 2018, 02:41:10 am »
Sounds like a good benchmark! I'll sneak that in as a "psychedelic" mod :D

The Lua context(which holds the memory for the Lua scripts) will initialise only with function pointers to allow the user to CRUD(create, read, update, delete) game objects. This means we can limited things to just "read only" if we want, but also means that no game state information will be held in the Lua context itself.

but Lua shroud obey game objects life cycle, if some object right now is created by function A then Lua should use this function too to create object. Same with deleting. If something is not changeable by game engine scripts usually should not change this too, this is important in case when some game logic really on that value is not changed.


As mentioned above, we can limit access to read only since nothing will be stored in the Lua context.

Best if you could control per script call, some scripts could be allowed to change unit in any way other can only read.


Why is this a requirement? I'm not trying to replace yankesScript, at least, not initially. I'm only trying to provide a mod(der) friendly interface for other people to make what they want.

https://www.youtube.com/watch?v=sqcLjcSloXs
Overall only thing that need to work is scripts that are used by biggest mods there. And it would be good that after transition they do not loose any functionality.

Might need to define "too much" :D

I'll have to change the interface for loading/saving objects(almost everything in SaveGame, and most things that already take YAML node as a parameter) to include the lua context so that I can pass data to/from the game state.

You can compare it with current scripts, if your changes will be deleting my function parameter and replacing by your then its 100% fine. But if you need change every game logic class then is not fine.

Cool!

small clarification, this was condition not statement, you should read it "[if] Meridian is fine too to change script engine."

I don't think this'll be an issue

"Hello World" isn't enough :P

I'm compiling a list of "example mods" as a way of specifying a Minimal Viable Product(MVP) milestone. So far, I have the following:

* add new field to the soldier for mana, and show how that can be used to implement new gameplay mechanics
* modify world zones and restrict missions in the zones to allow for diplomacy as a new mechanic
* implement "traitors" which are enemies that flip sides during battle and join your team
* open for more suggestions!

This could be good start.

Like I said, the goal is not to kill yankesScript, but to supplement it initially. If there comes a point where the two collide, then yeah, we may want to revisit this strategy, but otherwise I don't see a problem with them co-existing :D

Whole point is that I'm not interesting in maintaining two script engines in OXCE (and probably Meridian too) this is way I set high-bar that new script engine is objective better in every aspect compare to my engine. Otherwise I will stick to my creation :) This mean you will need do more work but I would be unwise to give up couple of years of work for free :>

[ps]

Instead of wiki I would be more interested in WiP commits that show how it will work.
« Last Edit: September 17, 2018, 02:43:00 am by Yankes »

Offline SupSuper

  • Lazy Developer
  • Administrator
  • Commander
  • *****
  • Posts: 2162
    • View Profile
Re: [OXCE+] Lua Integration
« Reply #14 on: September 17, 2018, 06:49:23 am »
Figured I'd throw my two cents in, because I've always been curious about integrating Lua scripting (I've played with it in other games), but have no idea how. :P Usually when people talk about scripting they think about modders doing anything, limitless possibilities! I don't think that's the point. OpenXcom is already open-source, the possibilities are already limitless, so clearly it's not that simple. At the end of the day, modders are modders and programmers are programmers and I don't think scripting is what will convert them. But in my eyes it has the potential to help solve one of OpenXcom's biggest problems: maintainability. Now that vanilla is basically done, all that's left is adding modding features, forever and ever...

Everytime a mod needs a feature, a modder has to convince a programmer to add that feature, and that feature is then available to every single mod. This poses multiple problems. Every feature has to be compatible with every other feature, and as mods grow, so will the engine. Features are never removed, only added, and the weight is on every programmer that follows. Every feature, no matter how specific and particular, has to be generalized to work for everyone. Etc. Does it make sense for the weight of every mod to be on every other mod? On every programmer? Probably not. It's a maintainability and scalability nightmare, and in the end programmers have to be choosy and decide what is worth supporting or not, fragmenting the modders and codebases.

So, instead of the engine having to support every feature for every mod, what if mods could carry their own code? Their own features? Instead of Items needing 100+ properties, what if they could instead have scriptable functions for onFire, onUse, onHit, etc. This would both give mods more power and make code lighter and easier to maintain for programmers. Now I don't think this means exposing everything and creating a completely generic scriptable engine, that's its own nightmare. Just exposing the most common "game events" from which modders can manipulate the simulation would suffice, letting them add their own associated state and UI if needed. A happy middleground between modder potential and programmer scope.

So why Lua vs Yankes scripts? Well it's pretty clear adoption has not been great, modders would rather bug programmers than deal with it, and the only programmers that deal with it are... Yankes. And I think a big factor is because modders aren't programmers. You can give them all the code and potential in the world, if they don't know how to use it, it's useless. So will they code Lua? Probably not. The draw of scripting isn't to the modders who can't code, but to the programmers who can't code C++ but can deal with something simpler like Lua, which in turn help the modders. I doubt they're itching to learn a whole new bespoke scripting language.

As for performance... honestly, you could do a lot worse than Lua. It's the most common lightweight scripting language around, designed for embedded, runs on every platform on the planet. It'll never beat native, it's a bytecode VM, but for anything outside a tight game loop it should be fine. After all, OpenXcom struggles on anything under a 2Ghz and does all its graphical operations on the CPU, so I don't think we're winning any awards for performance either. :P
« Last Edit: September 17, 2018, 06:52:25 am by SupSuper »