When enemies are wielding explosive or incendiary weapons, it seems they don't care if they're standing within the blast radius. This is problematic because, well, from time to time they suicide this way. Note that this is not a problem in vanilla games, since enemies don't really get a large variety of such weapons there.
I'm not looking for a complicated solution, but:
If an enemy has a choice of using two weapons, they should prefer to use the one with which they won't cause substantial damage to themselves.
It recently happened to me that an enemy had a knife and a molotov cocktail. I was standing in melee range of them. So what did they do? They blasted me and themselves with molotov, instead of killing me with a knife lol. So a simple solution would be to prefer a melee weapon in melee range, if the other choice is a weapon with a blast radius.
A more complicated solution would involve blast radius and damage calculation on all the units caught in the blast radius, and to only use the blast radius weapon if the damage on friendly units would be a factor lower than the damage done on enemy units (either because of resistances, or radius damage dissipation). Could also include a random chance to just go and blast anyway.
An even more complicated solution would also involve moving away from the target until the calculation would be favorable enough.
But yeah, I think the first simple melee solution is good enough.