On a side note, I remember in vanilla UFO that battleships that were definitely attacking headed to your base at full speed. Not here in OpenXcom, it's around half speed instead. Was it an intended change?
Conventional radars have 10%(small)/20%(large) chance to detect and scan every 30 minutes. Retaliation battleships are flying with speed of 2700knots, and large radar detection distance is 2577 miles, so radar manages to scan almost 2 times. To be precise - battleship covers large radar range in 57 minutes. First time let say be LARGE (20%) and second time is SMALL radar, thus 10+20 = ~28%. You can be unlucky and get only ONE scan. In this case chances will be ~25%. Since it's very large ship which is flying very high - there's a chance modifier +20%. So it will be 30%.
According formulas of theory of probability for "at least one probable event": Chance = 1 - (1-chance1)*(1-chance2) = 1-(1-0.24)*(1-0.36) = 0.5136 = 51%.
So roughly each one of two battleships manages to get to the base without being noticed by radars. Unless there is a hyberwave decoder (it will always get its 100% chance).
That how it should work.And it's always RNG issue.
If you want to disprove probability ratio, you can make a save, put "retaliationTarget: true" to all your bases (to make all bases revealed to aliens), and in the alienMissions section set "spawnCountdown: 1" for the retaliation. And then load this save, like, 100 times, and count real %% ratio (after putting saveScumming in the advanced options to ON).
I've made a check for the base with single large radar (0 is failed to detect, 1 is successifully detected).
1000100010 0010100011
0001100111 0101001100
0001101001 1111001001
So experiment proves 26 successiful detections from 60 tries, which means ~43% chance.
Theory: with single large radar it will be two scans with chance 24% (considering size and altitude). 1-(1-0.24)*(1-0.24)=1-0.5776 = 0.42= 42%
So probability formula (theory) is in perfect line with facts.
It's bad idea to make any conclusions from pretty random events. Until you gather any weighty statistics.
Stop doubting in OpenXcom