Random thoughts on external AI

[Dioxine]

EDIT/Meridian: Split from original topic: https://openxcom.org/forum/index.php/topic,6580.0.html

With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?

[Whispers]

With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?

So are you looking to completely replace the AI, or merely hook in to the existing AI and tweak things?

@Whispers: called it

Yes, yes you did. No, this will not allow you to do any networking at all. Networking is my day job and I'm not sure I want to spend my evenings doing my day job... unless you guys pay me... A lot of money 

[Meridian]

With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?

Theoretically yes, although we don't need Lua for that... I could make a call to an external engine and pass the entire savegame as a parameter in C++ too... wouldn't take more than a day. If we count also parsing the response from the service and acting on it, probably 2-3 days.

But:
1/ Who is going to implement that neural network?

2/ I have never done a neural network, but I understand some basic principles. It would probably take a few years to implement just the input-creating program, nothing to do with neural networks or machine learning yet (parse the whole savegame, digest it and prepare a simplified input for the neural network). Assuming preparing such simplified input is even possible (without losing too much detail, or sanity), add a few more years to implement (or better said invent) a suitable neural network.

3/ Once you have that, add about a 100 years, give or take a decade, to train it.

4/ Once trained to process the input and make a decision in let's say 1 second... happy waiting each time it is called (several hundred or thousand times per turn)

Jokes aside, it took a decent sized team of world class scientists several years to create AI that can play poker well enough to beat human players (beat = win in a bit more than 50% of cases).
I'd guesstimate that Xcom is about a million times more complicated than poker.

I don't have any numbers on how many hundreds/thousands of people worked on self-driving cars for how many decades... but an engine capable of solving Xcom would not be simpler than a self-driving car either. Good luck

PS: unless -- as @Whisper says -- you just want to take existing partial decision at some point of decision making (e.g. just before rolling dice on x% attack, y% patrol, z% ambush and w% retreat... which is the tightest bottleneck in the process I can think of) and tweak those 4 numbers by some magic (or neural network) into a different set of four numbers... but I guess RNG does a good job at that, so you'll probably want a slightly more complicated breakpoint

PPS: in one sentence: A scripting engine (be it Y-scripts or Lua-scripts or anything else) allows you
- to pause the program's execution (at points decided by the script engine creator),
- access the program's variables (only those that the script engine creator gives you),
- do some math using some language on those variables (that's the only difference between Y-scripts and Lua-scripts),
- and return some modified variables to the program... after which it will resume its execution (btw. the variables you can return are also just what the script engine creator gives you).

[Dioxine]

Theoretically yes, although we don't need Lua for that... I could make a call to an external engine and pass the entire savegame as a parameter in C++ too... wouldn't take more than a day. If we count also parsing the response from the service and acting on it, probably 2-3 days.

But:
1/ Who is going to implement that neural network?

I know someone interested. (He does not wish to be named. I refer to him as "Techpriest"). But thanks for the lengthy answer - I will pass all of it to that guy.

[KZad Bhat]

For someone with the knowledge of how to design and build effective neural networks, and willing to take the time, wouldn't they be better served using it to find more effective ways to code field assessment and other variables to make a smarter AI in machine code? The major advantage to using a neural network "live" is that if properly trained beforehand, the computer AI would then be able to keep learning, continually adapting to the player, with the disadvantage of really sucking up CPU and memory resources to do it. I can see that increasing the minimum system specs to be able to actually play the game from the current early-mid '00s era computer to something high-end today.

[Meridian]

I know someone interested. (He does not wish to be named. I refer to him as "Techpriest"). But thanks for the lengthy answer - I will pass all of it to that guy.

This guy?

[Stoddard]

re AI: Here's how to defeat it with elephants: https://www.quantamagazine.org/machine-learning-confronts-the-elephant-in-the-room-20180920/

[Whispers]

Getting a little off topic here, but I don't see a technical problem building a neural network, although it would take a lot of effort.

Step 1) Gather as many events and store every action that a user, and the existing AI take
Step 2) Use that to train a generative neural network to simulate both the user and the existing AI.
Step 3) Take both AIs and pit them against each other(adversarial generative neural network).
Step 4) Let it run continuously for a looooooong time. Meanwhile, keep feeding it new playthroughs and more data. Maybe pausing it every week and playing it to make sure it is on the right track.

The biggest problem I see is that you would have to build your network in Lua. Problem with that is that there are so many other better tools out there for building deep learning networks and, well, it would be hard to compete with that. You could, with source, make it so that you can use Tensorflow or one of the other machine learning toolkits, so, to get back on topic and answer the original question...

If you know how to do all the above, then yes, you can build a neural network AI that would obliterate even the best players, but if you know how to do all the above, then I doubt you would want to use Lua to do it. So, yes, you could, but why?

[Meridian]

It sounds really good on paper... but it is practically impossible in my opinion.

Let's look just on step 1 for starters.
In poker, events (card dealing) and actions (check, call, bet, raise, fold) are enough... not so in xcom... in xcom you need a snapshot of the entire battlescape after each move, because any events and actions directly depend on that. Unless you want to train it only on one single fixed map, like chess

As an example, it takes 100 (input) neurons for image recognition of a 10x10 picture.
By analogy, on an average 60x60x8 piratez map, it would take 60x60x8x16x16x24 ~ 176 million input neurons... just to know where solid matter is located on the map.

Steps 2/3:
Does that mean unsupervised machine learning? With such enormous input size and variation, it will take longer than the current age of the universe... until it starts making first baby steps.

Again, unless I am completely wrong, and please correct me if I am... nobody ever attempted anything even remotely as complicated as this. So unless someone can drastically simplify the xcom model to work with... it will remain just a dream.

[ohartenstein23]

If nothing else, the network could be simplified to just make a more informed choice on what the AI already does. Have the inputs be number of spotted X-Com units, the unit's intelligence and aggression, the unit's current HP, and maybe add in if any friends are nearby, then output the decision of fight, item, run trying to take a shot, setting up an ambush, trying to escape, or scouting. Sure it'd be nothing special in terms of massive behavioral improvement, but it'd be simpler to implement as a proof of concept.

[Solarius Scorch]

I think at this point we should be speaking about whether this feature will allow modifying the AI in general rather than specifically about neural networks. NN are a possible technical solution, but not the only one.
And X-Com AI would definitely could use some work (though it's been improved already, mainly by Otto Hartenstein).

[Meridian]

I think at this point we should be speaking about whether this feature will allow modifying the AI in general rather than specifically about neural networks. NN are a possible technical solution, but not the only one.
And X-Com AI would definitely could use some work (though it's been improved already, mainly by Otto Hartenstein).

This feature will not help, nor hinder improvement of xcom AI by external engines.

As I said, AI can be externalized (with or without using this feature) with a relatively small effort.

Big effort is:
1. to even decide what information to send to the external engine; and where to make breakpoint(s) for external call(s)
2. implement external engine

PS: if you want internal AI improvement (as opposed to external), this feature is (practically) equal in power and possibilities to Y-scripts

[Yankes]

It sounds really good on paper... but it is practically impossible in my opinion.

Let's look just on step 1 for starters.
In poker, events (card dealing) and actions (check, call, bet, raise, fold) are enough... not so in xcom... in xcom you need a snapshot of the entire battlescape after each move, because any events and actions directly depend on that. Unless you want to train it only on one single fixed map, like chess

As an example, it takes 100 (input) neurons for image recognition of a 10x10 picture.
By analogy, on an average 60x60x8 piratez map, it would take 60x60x8x16x16x24 ~ 176 million input neurons... just to know where solid matter is located on the map.

Steps 2/3:
Does that mean unsupervised machine learning? With such enormous input size and variation, it will take longer than the current age of the universe... until it starts making first baby steps.

Again, unless I am completely wrong, and please correct me if I am... nobody ever attempted anything even remotely as complicated as this. So unless someone can drastically simplify the xcom model to work with... it will remain just a dream.

I bit disagree ablaut  176 million input neurons, LoF can be done by code and network will get only Y/N info per node. I could even say lot less if network is only responsible for high decisions making not micro management. Overall AI can play Dota2 and Starcraft2 and same is possible for OXCE. Only problem is resources, until Google will backed us, ETA of this feature will be 100 years

[Meridian]

I bit disagree ablaut  176 million input neurons, LoF can be done by code and network will get only Y/N info per node. I could even say lot less if network is only responsible for high decisions making not micro management. Overall AI can play Dota2 and Starcraft2 and same is possible for OXCE. Only problem is resources, until Google will backed us, ETA of this feature will be 100 years

Of course!

I was describing the "no-effort-on-openxcom-side-let-external-engine-do-everything" solution.

If we want openxcom actively take (a very significant) part on all this... then we are back at the beginning... and we can just implement it in openxcom, without scripting and without external engines.

[KZad Bhat]

The neural network doesn't need to deal with individual pixels, merely objects and the details of those objects. So you won't need 176 million input neurons. Though you would need input neurons for each space . . . 60x60x30 being the biggest map, you'd need 108,000 for that portion, though then each needs to recognize what object is there, what facing that object has (if any), what objects it can see, and what objects can see it. Although maybe more efficient, and certainly more fitting with developing an improved set of AI algorithms, is to have each actor be an input, basing everything on what that node can see.

Beyond that, though, I couldn't add much. I've only directly seen the results of one neural network, and I had nothing to do with its construction so I don't even know what went into it.