my first post!
I'm a recent convert to OpenXcom and I have been livestreaming it on Twitch/Hitbox via OBS.
Window Capture works for most things, but if it suddenly doesn't, you can try Game Capture and see if it works differently. Monitor Capture is theoretically also an option if you're recording fullscreen, but I don't know how it deals with a multi-monitor setup. (It might actually be easier, because you can put OXC on one monitor, but borderless windowed, and everything else on the other.)
Part of why I appreciate OBS so much is that it's got a couple of options, even though that makes troubleshooting more complex.
My current Xcom/OBS config has OXC set to desktop resolution, Borderless, no OpenGL filters, and OBS set to Window Capture. The most common things you have to look for in trying to make these two work together, imo, are the different types of capture in OBS, and possibly disabling Aero if you're on Windows and non-fullscreen captures refuse to work.
It's also worth noting that if your system is beefy enough, you can record 720p and set the encoding bitrate to a whopping 5000k and YouTube will generally accept it for HD playback. (Or, if you livestream on Twitch at 720p, it will export to YouTube as HD regardless of your encoding bitrate.)