I've seen a few people mention this, and it was something I wanted for myself, so I put this together. It extends the spawn time of all slaves and workers (except Yeoman) in hideout defense by 10 half-turns, so you shouldn't see any of them spawning until after your 5th turn. It's pretty easy to modify yourself to your own preferences with the relevant values isolated here.
It's a bit hacky, since I had to replace data directly in the base rulesets, but seems fully functional. I don't think it would cause any problems, but can't be sure of that. Keep in mind that this will only work on hideout defenses started after the mod is applied, since it runs during the battle generation.