Is there some sort of breakdown or explanation of how the combat routine works? I've not found any kind of lay and explanation yet.
The following is my interpretation from the code at AlienBAIState.cpp. I'm definitely not a code expert, so I might be wrong in some conclusions. The most important aspects to take in mind are the alien's Intelligence, which determines the amount of turns it can remember an enemy's position even without seeing it, and its Aggression stat, which sets how it will respond upon seeing an enemy (0 = more likely to pull back and set a reaction ambush, 2 = more likely to attack rightaway).
There are 5 AI modes:
* Patrol
* Ambush
* Combat
* Retreat
* Psi
There are several conditions that will determine which mode the AI will select for each unit, including sometimes a random element. An alien's Intelligence and Aggression will be a major factor for certain modes, including the type of weapon it is carrying (rifle vs melee).
Patrol is used when an alien doesn't remember the position of any enemies, nor it can see any enemies, either looking back at him or not). When determining where to patrol the AI considers the 'flag' that defines a route node's importance. If an alien spots or learns of an enemy's location it will then reevaluate its next action. Aggression is also a factor when choosing the reaction shot type when on Patrol: Aimed is used by low aggression units, while Auto will be chosen for high aggression.
There's too many conditions for each specific mode, plus more conditions for when choosing the mode, to list them all here.