Reading your impressions is interesting, as I'm probably in a similar category of player. I discovered the X-Com Files a few months ago, and have been playing heavily since. I've been playing on "Experienced" difficulty, and am on my third try after abandoning my first two plays due to various deficiencies in approach as I was learning the ropes. I feel my current game is going quite well, so maybe I've figured a few things out finally.
I mostly agree with your impressions of weapon usefulness. My agents usually stick with magnum pistols, shotguns, and stun rods, at least until aliens start their invasion proper. Some of the rifles then become more useful, such as laser and chemical/toxic varieties. Tritanium ammo in smart magnums keeps pistols highly useful and relevant quite far into game, and agents with high marksmanship can use them at pretty good ranges. I don't think I've even used ten percent of all the varieties of weapons found in the game. Most of them don't seem useful to me, and most of them are virtually indistinguishable.
Looking over your comments as a whole, my general deduction and friendly advice is that you need to do a lot more missions-- and on those missions, you need to capture more enemies alive. Missions generate research topics, and apart from developing some combat tactics that work for you research is the most fundamentally important part of this game. Your comments that you feel you're caught up on research tell me you probably aren't really aware of the vast scope of things to be investigated. This is why I say you need to do a lot more missions-- if you've succeeded in researching all the topics available to you, then the problem is that you're missing all kinds of stuff. For example, you should have had cyber armor in ample quantities (it takes a long time to produce) before the aliens started arriving en masse.
I've made research my top priority in my current game, which as I said is going fairly well. My game is currently in August of 1999, I have 130 scientists on the payroll, and I'm not at all nor have I at any point been in a position I'd describe as "caught up" on research. There are hundreds and hundreds of topics to research, all of which contribute directly to your warfighting capability. Also, a lot of interesting topics only become available by researching multiple instances of particular enemies.
I'm not in a position to say whether the way I'm playing is a good way to play, I can only say it seems to be working well for me. I've got seven bases, with over 100 agents in the field doing every mission which comes up and which I can get to in time. Two of these bases are mostly air bases, housing multiple fighter craft for shooting down ufos. Two others are mostly logistical or storage centers: one a warehouse for alien tech, the other a warehouse for everything of earthly origin.
This means I have 3 bases from which almost all operations are launched. The first base houses all the rookies. These folks only go on missions related to monsters, because most monsters don't shoot back. Rookies stay in this base, going on missions and training in the gym, until they get a "training complete" sign-off meaning they've maxed their stats as much as possible through training in the gym. About half my roster is in this base right now. After the 100th time hunting down beetles or rats or zombies, these missions frankly grow kind of tedious-- but it's vital to keep on doing them, as often as possible, because this is where the agent development occurs that allows you to face the bigger challenges of the game.
The second mission-responding base I call the Criminal Investigative Division. Agents at this base only fight cults, gangs, or terrestial paramilitary groups. During this process, they continue to gain competence with combat-gained stat increases. I still have these people using Tritanium vests, and they're getting along just fine against the kinds of enemies they routinely face.
Only once agents are at or near the actual stat caps ("super agents" as some might call them) do they move on to postings at the other five bases. This could be guard duty or fighter pilot duty at the airbases or logistical centers. Or, they could get stationed at what I'm calling the Alien Research Division-- in other words, the soldiers who fight aliens or Dimension X foes. That is the sole purpose of the third "fighting" base I have.
The point is, this is a logical way to organize soldier development. At each stage, agents are only fighting enemies whom they are capable of overcoming at that time. Only the toughest and most capable are fighting aliens, and this system seems to work well in producing a steady flow of such soldiers. Also, agents stationed at any base except for the rookie base are all in psionic training.
Furthermore, this system has the virtue of simplicity from a player's perspective. If a certain kind of mission appears, any soldier at the base responsible for handling those kinds of missions can respond. There's no need to spend time considering who is assigned to various crafts operating out of that base.
At the present time in my current game (August 1999), I'm mainly fighting aliens and the Syndicate. It's been quite some time ago that I finished off the main cults (not counting Cult of Apocalypse) and got rid of most of the monsters (not counting high-level zombie bosses, who continue to generate missions).
About the catastrophic loss of your base during alien retaliation, some would surely disagree but I would say in a case like this, just use debug mode and give yourself a win in that battle. It's not a competitive game-- the only objectives are to have fun, and hone your skills. X-Com Files is a very big game, and one shouldn't feel any scruples over occasionally giving onself a helping hand to continue getting a return (in fun) on all the time invested in playing.
Also as tkzv noted, though it's sometimes painful or sad, get rid of those low psi-resistant people as soon as you find out about them. They're a menace to themselves and the rest of your people, so they just have to go back to catching bank robbers or whatever else they were doing before.