OpenXcom Forum
Modding => Work In Progress => Topic started by: Dioxine on September 22, 2018, 09:19:09 pm
-
EDIT/Meridian: Split from original topic: https://openxcom.org/forum/index.php/topic,6580.0.html
With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?
-
With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?
So are you looking to completely replace the AI, or merely hook in to the existing AI and tweak things?
@Whispers: called it
Yes, yes you did. No, this will not allow you to do any networking at all. Networking is my day job and I'm not sure I want to spend my evenings doing my day job... unless you guys pay me... A lot of money ;D
-
With this Lua support, would it be possible to affect the AI decisionmaking by outside software (like neural networks)?
Theoretically yes, although we don't need Lua for that... I could make a call to an external engine and pass the entire savegame as a parameter in C++ too... wouldn't take more than a day. If we count also parsing the response from the service and acting on it, probably 2-3 days.
But:
1/ Who is going to implement that neural network? :)
2/ I have never done a neural network, but I understand some basic principles. It would probably take a few years to implement just the input-creating program, nothing to do with neural networks or machine learning yet (parse the whole savegame, digest it and prepare a simplified input for the neural network). Assuming preparing such simplified input is even possible (without losing too much detail, or sanity), add a few more years to implement (or better said invent) a suitable neural network.
3/ Once you have that, add about a 100 years, give or take a decade, to train it.
4/ Once trained to process the input and make a decision in let's say 1 second... happy waiting each time it is called (several hundred or thousand times per turn)
Jokes aside, it took a decent sized team of world class scientists several years to create AI that can play poker well enough to beat human players (beat = win in a bit more than 50% of cases).
I'd guesstimate that Xcom is about a million times more complicated than poker.
I don't have any numbers on how many hundreds/thousands of people worked on self-driving cars for how many decades... but an engine capable of solving Xcom would not be simpler than a self-driving car either. Good luck :)
PS: unless -- as @Whisper says -- you just want to take existing partial decision at some point of decision making (e.g. just before rolling dice on x% attack, y% patrol, z% ambush and w% retreat... which is the tightest bottleneck in the process I can think of) and tweak those 4 numbers by some magic (or neural network) into a different set of four numbers... but I guess RNG does a good job at that, so you'll probably want a slightly more complicated breakpoint
PPS: in one sentence: A scripting engine (be it Y-scripts or Lua-scripts or anything else) allows you
- to pause the program's execution (at points decided by the script engine creator),
- access the program's variables (only those that the script engine creator gives you),
- do some math using some language on those variables (that's the only difference between Y-scripts and Lua-scripts),
- and return some modified variables to the program... after which it will resume its execution (btw. the variables you can return are also just what the script engine creator gives you).
-
Theoretically yes, although we don't need Lua for that... I could make a call to an external engine and pass the entire savegame as a parameter in C++ too... wouldn't take more than a day. If we count also parsing the response from the service and acting on it, probably 2-3 days.
But:
1/ Who is going to implement that neural network? :)
I know someone interested. (He does not wish to be named. I refer to him as "Techpriest"). But thanks for the lengthy answer - I will pass all of it to that guy.
-
For someone with the knowledge of how to design and build effective neural networks, and willing to take the time, wouldn't they be better served using it to find more effective ways to code field assessment and other variables to make a smarter AI in machine code? The major advantage to using a neural network "live" is that if properly trained beforehand, the computer AI would then be able to keep learning, continually adapting to the player, with the disadvantage of really sucking up CPU and memory resources to do it. I can see that increasing the minimum system specs to be able to actually play the game from the current early-mid '00s era computer to something high-end today.
-
I know someone interested. (He does not wish to be named. I refer to him as "Techpriest"). But thanks for the lengthy answer - I will pass all of it to that guy.
This guy?
-
re AI: Here's how to defeat it with elephants: https://www.quantamagazine.org/machine-learning-confronts-the-elephant-in-the-room-20180920/ (https://www.quantamagazine.org/machine-learning-confronts-the-elephant-in-the-room-20180920/)
-
Getting a little off topic here, but I don't see a technical problem building a neural network, although it would take a lot of effort.
Step 1) Gather as many events and store every action that a user, and the existing AI take
Step 2) Use that to train a generative neural network to simulate both the user and the existing AI.
Step 3) Take both AIs and pit them against each other(adversarial generative neural network).
Step 4) Let it run continuously for a looooooong time. Meanwhile, keep feeding it new playthroughs and more data. Maybe pausing it every week and playing it to make sure it is on the right track.
The biggest problem I see is that you would have to build your network in Lua. Problem with that is that there are so many other better tools out there for building deep learning networks and, well, it would be hard to compete with that. You could, with source, make it so that you can use Tensorflow or one of the other machine learning toolkits, so, to get back on topic and answer the original question...
If you know how to do all the above, then yes, you can build a neural network AI that would obliterate even the best players, but if you know how to do all the above, then I doubt you would want to use Lua to do it. So, yes, you could, but why?
-
It sounds really good on paper... but it is practically impossible in my opinion.
Let's look just on step 1 for starters.
In poker, events (card dealing) and actions (check, call, bet, raise, fold) are enough... not so in xcom... in xcom you need a snapshot of the entire battlescape after each move, because any events and actions directly depend on that. Unless you want to train it only on one single fixed map, like chess :)
As an example, it takes 100 (input) neurons for image recognition of a 10x10 picture.
By analogy, on an average 60x60x8 piratez map, it would take 60x60x8x16x16x24 ~ 176 million input neurons... just to know where solid matter is located on the map.
Steps 2/3:
Does that mean unsupervised machine learning? With such enormous input size and variation, it will take longer than the current age of the universe... until it starts making first baby steps.
Again, unless I am completely wrong, and please correct me if I am... nobody ever attempted anything even remotely as complicated as this. So unless someone can drastically simplify the xcom model to work with... it will remain just a dream.
-
If nothing else, the network could be simplified to just make a more informed choice on what the AI already does. Have the inputs be number of spotted X-Com units, the unit's intelligence and aggression, the unit's current HP, and maybe add in if any friends are nearby, then output the decision of
fight, item, run trying to take a shot, setting up an ambush, trying to escape, or scouting. Sure it'd be nothing special in terms of massive behavioral improvement, but it'd be simpler to implement as a proof of concept.
-
I think at this point we should be speaking about whether this feature will allow modifying the AI in general rather than specifically about neural networks. NN are a possible technical solution, but not the only one.
And X-Com AI would definitely could use some work (though it's been improved already, mainly by Otto Hartenstein).
-
I think at this point we should be speaking about whether this feature will allow modifying the AI in general rather than specifically about neural networks. NN are a possible technical solution, but not the only one.
And X-Com AI would definitely could use some work (though it's been improved already, mainly by Otto Hartenstein).
This feature will not help, nor hinder improvement of xcom AI by external engines.
As I said, AI can be externalized (with or without using this feature) with a relatively small effort.
Big effort is:
1. to even decide what information to send to the external engine; and where to make breakpoint(s) for external call(s)
2. implement external engine
PS: if you want internal AI improvement (as opposed to external), this feature is (practically) equal in power and possibilities to Y-scripts
-
It sounds really good on paper... but it is practically impossible in my opinion.
Let's look just on step 1 for starters.
In poker, events (card dealing) and actions (check, call, bet, raise, fold) are enough... not so in xcom... in xcom you need a snapshot of the entire battlescape after each move, because any events and actions directly depend on that. Unless you want to train it only on one single fixed map, like chess :)
As an example, it takes 100 (input) neurons for image recognition of a 10x10 picture.
By analogy, on an average 60x60x8 piratez map, it would take 60x60x8x16x16x24 ~ 176 million input neurons... just to know where solid matter is located on the map.
Steps 2/3:
Does that mean unsupervised machine learning? With such enormous input size and variation, it will take longer than the current age of the universe... until it starts making first baby steps.
Again, unless I am completely wrong, and please correct me if I am... nobody ever attempted anything even remotely as complicated as this. So unless someone can drastically simplify the xcom model to work with... it will remain just a dream.
I bit disagree ablaut 176 million input neurons, LoF can be done by code and network will get only Y/N info per node. I could even say lot less if network is only responsible for high decisions making not micro management. Overall AI can play Dota2 and Starcraft2 and same is possible for OXCE. Only problem is resources, until Google will backed us, ETA of this feature will be 100 years :D
-
I bit disagree ablaut 176 million input neurons, LoF can be done by code and network will get only Y/N info per node. I could even say lot less if network is only responsible for high decisions making not micro management. Overall AI can play Dota2 and Starcraft2 and same is possible for OXCE. Only problem is resources, until Google will backed us, ETA of this feature will be 100 years :D
Of course!
I was describing the "no-effort-on-openxcom-side-let-external-engine-do-everything" solution.
If we want openxcom actively take (a very significant) part on all this... then we are back at the beginning... and we can just implement it in openxcom, without scripting and without external engines.
-
The neural network doesn't need to deal with individual pixels, merely objects and the details of those objects. So you won't need 176 million input neurons. Though you would need input neurons for each space . . . 60x60x30 being the biggest map, you'd need 108,000 for that portion, though then each needs to recognize what object is there, what facing that object has (if any), what objects it can see, and what objects can see it. Although maybe more efficient, and certainly more fitting with developing an improved set of AI algorithms, is to have each actor be an input, basing everything on what that node can see.
Beyond that, though, I couldn't add much. I've only directly seen the results of one neural network, and I had nothing to do with its construction so I don't even know what went into it.
-
I already answered, that if OpenXcom does the hard job of precalculating everything for the neural network then yes, it won't need that many.
The neural network doesn't need to deal with individual pixels, merely objects and the details of those objects.
Don't "details of those objects" include each voxel (and more)? :)
though then each needs to recognize what object is there, what facing that object has (if any), what objects it can see, and what objects can see it.
Which it could either do on its own, based on those 176 million voxels...
... or let OpenXcom do the hard job for it.
PS: I'm not saying this thing is unthinkable... I'm saying that it inevitably requires a LOT more effort on OpenXcom side than "just calling" some random external engine.
EDIT: I've split the discussion into a separate topic
-
They way most modern neural networks work is by defining training data and then building networks of decision trees. I've dabbled in some tensorflow network designs, and it takes a long time to get used to how it works, and get your mind wrapped around a lot of the concepts. However, someone with some experience could easily build a network in a few months time.
At it's most basic level, the neural network has a series of points to spend. Time Units are the currency of the game in Battlescape. Because of which, it actually becomes easier to define the network. You can spend those points by moving, attacking, throwing, or various other things, but it's still just a currency. Now, selecting the right thing to spend that currency on is the trick. First, you develop a hierarchy of networks. One for scouting(when no enemies are seen), another for attacking(for when an enemy is spotted), and maybe another one for retreating(for when forces are too strong), and perhaps a fourth one for regrouping. You train each of the sub-networks in isolation, then combine them for a more strategic AI that oversees it all.
For scouting, you define the success of the network by it's ability to find each enemy, and weigh that against defensive metric(have they ended up in a spot with a good defence ability, like behind a wall or in some smoke). For attacking, the success criteria is a lot easier, kill everything with minimal damage to your own units. For retreating, you have to get your units back to a point without killing them. And finally, for regrouping, the success depends on the ability to get everyone to a point with some defensive ability. Once you've trained each of the individual AIs, then you can easily train a strategic AI.
All of this is months and months of work. I won't have time to do this, but someone with some more interest in this could do it.
-
https://www.youtube.com/watch?v=GOFws_hhZs8
Probably good introduction of genetic algorithms that are closely related to networks and AI.
-
No, you do not need information on each individual voxel to give that information. In fact, all of those states can be the same, but if you apply a different image, you've not created a completely new set of inputs that are unnecessary, because nothing has actually changed as far as AI is concerned.
Yes, there could be ways of having OCX or its branches package the information in more useful ways, but off-hand the best I can think of is to report to the neural network the placement of objects, rather than the pixels representing them. Especially, if this is meant to develop a more effective AI algorithm, it's better to have the neural network processing exactly the information that the AI engine will be dealing with anyway. Since the AI engine is not looking at pixels/voxels (I really hope it isn't), no need to waste time having the network look at them either.
In fact, it may turn out that tying it each individual block and having those 108,000 neurons is not nearly the most efficient method of handling it. Perhaps an input just for each actor, or at most the visible blocks of each actor, again similar to however the AI engine already processes it. However the tech developing the network's processing sheets finds most effective, I don't know, but he almost certainly won't be looking at voxels, not for this part.
-
Especially, if this is meant to develop a more effective AI algorithm, it's better to have the neural network processing exactly the information that the AI engine will be dealing with anyway. Since the AI engine is not looking at pixels/voxels (I really hope it isn't), no need to waste time having the network look at them either.
The current AI spends 99% of the processing time looking at voxels, calculating LOS and LOF.
The actual decisions (after all voxel calculations prepared the possible options) are just a few ifs and a few dice rolls, taking zero time.
Yes, there could be ways of having OCX or its branches package the information in more useful ways, but off-hand the best I can think of is to report to the neural network the placement of objects, rather than the pixels representing them.
OpenXcom is not Xcom2012.... it cannot shoot through walls and solid objects.
Just a placement of an object is not enough to do anything in OpenXcom.