Mods tend to work together, but there is no guarantee. Usually, most mods which add new weapons and new races are compatible with each others to almost the full extent (races might interfere with each others a bit when it comes to researching the Alien Origins and other topics). You probably only need one mod that adds terrain and those are the worst for interfering in my experience, since they all tweak maps. The best mod to add more UFOs is "Yet More UFOs", which is compatible with the "Alien Armoury Expanded - Crafts" mod, for the best "lots of different UFOs" experience.
Overall, the best combination of mods for a "still close to vanilla" experience is no doubt the Final Mod Pack. A megamod combining a lot of mods into one and making sure everything is compatible. If you want a new experience, Piratez really delivers. It's pretty much a whole new game with the same engine.
I am working on the "XCom Armoury Expanded", to go with the "Alien Armoury Expanded" mod, to build a set of compatible, modular mods allowing you to define the experience you want.
I don't think there is anything mod-wise that controls the number of aliens at the exit of your craft, unfortunately. You do use smoke grenades, right? And even more useful, grab a mod with grenade launchers, fire a smoke grenade and it will explode instantly, allowing you to move now instead of waiting a turn. Or turn on the option for instant grenades and use regular smoke grenades (but that's a bit more weird).
Regarding psi, it's difficult.. Looking in the ruleset, you can find the psi-amp and "alien psi weapon" and tweak them. The "Psi Needs LoS" mod just adds the requirement for LoS (and it does it to the Mind Probe, making it quite useless imo). You can do other things (ex.: make TU cost higher so psi-power becomes more difficult to spam) if changing the LoS doesn't do what you want. I agree that the LoS thing nerfs it too much for aliens. Some people have had success defining a maximum range, I have not.
There is a "True LoS" mod out there, that requires you to use a different executable. Depending on your computer, keeping track of the full vision range may make the game slower though (not an issue for most, but if you have an old/crappy machine, it matters).
Ouf.. long answer..!