Hmmm.. As luck would have it, I've just done a clean High Sierra MacOS install here. (So am on 10.13.1 now.) Thought this would be a good chance to see how easy it is to run the nightlies. Note: the procedure below was exactly the same when I ran OpenXCom on MacOS 10.12 Sierra.
First one I tried was a nightly I grabbed back in 2016 (around February 2016, according to file dates) .. copied into my user Applications folder, and then did all the prep that I believe has to be done for all the nightlies (at least, for the binary versions -- I'm not sure if the App Builds have some of this self-contained):
Installed Home Brew from their
website .. ran the brew install commands from
here to get all the SDL components installed ("here" being where you get to after going to the
OpenXCom site -> Downloads -> Nightlies -> Apple -> Binary Builds).
So from there I was able to run my old early 2016 binary build by opening the Terminal, cd Applications, cd {your-game-folder}, ./openxcom to run it. ..but because SDL GFX is now a more recent version I got an error "Library not loaded: /usr/local/lib/libSDL_gfx.13.dylib .. so I get around that by cd-ing to /usr/local/lib/ and typing "cp libSDL_gfx.15.dylib libSDL_gfx.13.dylib" .. and then it works!
(With this old version I'd already copied my Steam XCOM game files into the UFO folder.)
But what we really want is the latest nightly. Unfortunately the zip, tar, and bz2 binary links from
OpenXCom site -> Downloads -> Nightlies -> Apple -> Binary Builds don't seem to work right now, so I went for the app version (
OpenXCom site -> Downloads -> Nightlies -> Apple -> App Builds) .. downloaded that, opened the disk-image, copied the app to the Applications folder I made in my home folder, right-click on it and "show package contents" and go into Contents -> Resources -> UFO and dump the contents of my Steam-installed XCOM folder into there. Jump back out to the main app icon, double-click on it and it all works beautifully. Wohoo!
(Oh, before running this newer build, I did go into ~/Library/Application Support/ and deleted the OpenXcom folder, because that was the preferences and saves from my early 2016 build .. always good to start clean.)
So, having the latest nightly app build working, I decided to also try going to the Forums -> OpenXCom Extended+ -> OXCE+ for MacOS .. grabbed the 2017-10-10-OXCEPlus-3.9c.dmg file and installed that. Same deal as the nightly .dmg file: copy it to the apps folder in my home folder, right-click and "show package contents", Contents -> Resources -> UFO and dump the contents of my Steam XCOM folder there. Jump back out to the main icon, double click and that works too. (Note: deleted the Library -> Applications Support -> OpenXcom folder before launching it, to have fresh settings and avoid inheriting config problems, of course.)
Once that worked, i grabbed my Terror from the Deep install from Steam and put that into the OXCE+ and vanilla nightly builds' Contents/Resources/TFTD folder too. Those tested okay too.
So yeah .. it works for me. You might want to try the homebrew install as instructed
here so it can install the YAML and SDL components. Maybe that's all you're missing?
If others that tried the App installs without having to do anything with Homebrew first, I'd be keen to hear it. This is the fourth wipe+install of MacOS I've done in the past week, and I'm not really keen to do it again just to see what components these nightlies *don't* need to run.
The one big problem I did have with the newest nightlies at first was that I was copying my XCOM files straight from my old early 2016 OpenXCOM install at first. That was crashing straight after the intro video. So I went back to Steam install game files, that hasn't been touched or patched in any way, and it went okay after that. {shrugs} Not sure what the problem was there.
So, I've typed lots of rambling text.. But hopefully it gives the Mac users here a bit of a chance of getting the nightlies to work for them. Documentation on now to do these on MacOS is a bit thin on the ground (or is often out-dated) when you look for it, so maybe this will help.