OpenXcom Forum

Contributions => Programming => Topic started by: Warboy1982 on May 07, 2015, 09:34:55 am

Title: Upgrading to the nightly
Post by: Warboy1982 on May 07, 2015, 09:34:55 am
as of today, the data folder structure will be drastically altered, and a new standard will be enforced for mods.

more details will appear in this thread, i just can't be bothered detailing them all yet.

full credit to myk002 for his work on this.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 07, 2015, 09:46:25 am
Overview:
In order to support both UFO and TFTD, we have to reorganize the OpenXcom directory structure a little bit.  Once this goes in, you'll have to reinstall OpenXcom from scratch, reinstall your UFO data files, reinstall your mods, and re-enable the mods in the Options/Mods screen.  And since we have to reinstall stuff anyway, we're taking the opportunity to change how mods are handled so it's easier to uninstall and upgrade them.

Details:

Screenshot: https://imgur.com/kqKpNsa


Step by Step Guide for Upgrading:

How to install a mod correctly:

Go to your mods directory, where your savegames are.  Extract the mod into a new subdirectory.  WinZip has an "Extract to" option that creates a directory whose name is based on the archive name.  It doesn't really matter what the directory name is as long as it is unique.  Some mods are packed with extra directories at the top, so you may need to move files around inside the new mod directory to get things straighted out.  For example, if you extract a mod to mods/LulzMod and you see something like:

    mods/LulzMod/data/TERRAIN/
    mods/LulzMod/data/Rulesets/

and so on, just move everything up a level so it looks like:

    mods/LulzMod/TERRAIN/
    mods/LulzMod/Rulesets/


Additional info for mod authors:

more mod metadata can be added easily later.  for example, we could introduce mod dependency and incompatibility information so mod authors can require other specific mods to be loaded before theirs or prevent two known incompatible mods from being loaded at the same time.


Step by step guide for updating your mod to make it ready for the new order:

First, let's consider a simple mod that consists of just a ruleset and a resource file.  Alternate Smoke is a good example of this: https://www.openxcom.com/mod/alternative-smoke
Inside the download, it looks like this:

    $ unzip -l alternative_smoke_v1.0.zip
    Archive:  alternative_smoke_v1.0.zip
      Length      Date    Time    Name
    ---------  ---------- -----   ----
            0  04-17-2015 10:04   data/
            0  04-17-2015 10:03   data/Resources/
            0  04-17-2015 10:04   data/Resources/AlternativeSmoke/
            0  04-17-2015 10:04   data/Resources/AlternativeSmoke/SMOKE/
         3750  03-24-2015 21:46   data/Resources/AlternativeSmoke/SMOKE/smoke_cloud.gif
            0  04-17-2015 10:04   data/Ruleset/
          227  04-17-2015 10:07   data/Ruleset/AlternativeSmoke.rul
    ---------                     -------
         3977                     7 files

let's condense that to:

    data/Resources/AlternativeSmoke/SMOKE/smoke_cloud.gif
    data/Ruleset/AlternativeSmoke.rul

Now, since mods no longer go in "data", but rather in their own directory under mods, let's chop off the top level so it looks like this:

    Resources/AlternativeSmoke/SMOKE/smoke_cloud.gif
    Ruleset/AlternativeSmoke.rul

Now users can extract the archive directly into the data directory for older OpenXcom installs or into a named subdirectory of mods for new OpenXcom installs.

Even though the new mod format allows rulesets to live in the top-level directory (i.e. AlternativeSmoke.rul instead of Ruleset/AlternativeSmoke.rul), let's leave it where it is for now in the Ruleset directory so it will work with older OpenXcom installs.  Later, once everyone is using a new OpenXcom build, mods can start just putting their ruleset files in the top level directory.

That is all that is minimally required, but if you want to see information about the mod when hovering over it on the Options/Mods page, you'll need to create a file named metadata.yml and add it to the root of your mod:

    metadata.yml
    Resources/AlternativeSmoke/SMOKE/smoke_cloud.gif
    Ruleset/AlternativeSmoke.rul

metadata.yml should have the following contents:

    name: "Alternative Smoke"
    version: 1.0
    description: "A less disturbing/distracting smoke animation."
    author: "Robin"
    master: "xcom1"


That's it!  For most mods, this is all that needs to be done.  The Final Mod Pack (https://www.openxcom.com/mod/final-mod-pack), for example, is already in the correct format -- that is, the MAPS, Resources, ROUTES, etc. directories are already at the top level.  It would just need a metadata.yml file like the one described above to make it ready.


Total conversion mods need a bit more work.  Let's take Piratez as an example (https://www.openxcom.com/mod/piratez).  It's metadata.yml file would look like this:

    name: "Piratez"
    version: 0.9h
    description: "The year is 2600. You run a gang of mutant pirates. Rob aliens and their human proxies for fun, profit and power."
    author: "Dioxine"
    id: piratez
    isMaster: true
    master: xcom1

This will cause Piratez to appear in the "Game type" list at the top of the mods page.  Other mods that modify Pirates can declare "pirates" as their master.  Note that Pirates would need a few other changes to be completely functional as a master, like unifying the sprite numbers across its rulesets.
Title: Re: Upgrading to the nightly
Post by: Meridian on May 07, 2015, 10:59:00 am
SWEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEET!!!!1111
Title: Re: Upgrading to the nightly
Post by: Jo5hua on May 07, 2015, 11:11:21 am
as of today, the data folder structure will be drastically altered, and a new standard will be enforced for mods.

more details will appear in this thread, i just can't be bothered detailing them all yet.

How will this affect the modsite? Should we add something to distinguish the mods, or just utilize the "Game Version Number"... Thats the minimum game version number field that people enter when they add mods (currently not showing on the public mod page).
Title: Re: Upgrading to the nightly
Post by: kkmic on May 07, 2015, 11:11:50 am
This was soooooooo needed

LE:
@Jo5shua: let things settle for a while, before even starting to think about what should be changed in the mod portal
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 07, 2015, 11:55:54 am
Just one question: When will this be implemented? I just wrote a webpage on how to compile and install openxcom on ubuntu, but with changed data-structure, I will have to rewrite that and recompile it as well. Would be nice to know, when this will take effect...
Title: Re: Upgrading to the nightly
Post by: Meridian on May 07, 2015, 12:01:56 pm
It says "as of today" in the first post, first sentence.
It is already implemented and ready to use.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 07, 2015, 12:08:49 pm
So, »effective immediately«. OK, work's coming up...
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on May 07, 2015, 12:49:35 pm
So, »effective immediately«. OK, work's coming up...

you might want to hold off on that, it's not completely finalized yet.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 07, 2015, 01:15:40 pm
I trust, there will be a notification here, when that has happend.
Title: Re: Upgrading to the nightly
Post by: Ishmaeel on May 07, 2015, 02:01:35 pm
Ah. Time to update my auto-download-'n-install script.

I've been dropping OXC directly into the Steam installation folder for client integration. I hope there will be provisions to specify the data folder location. May I report issues (if any) with that approach?
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on May 07, 2015, 02:23:10 pm
sure, stress testing would be appreciated at this point
Title: Re: Upgrading to the nightly
Post by: hellrazor on May 07, 2015, 03:09:18 pm
Just pulled this new awesome stuff from github.

I have already changed the directorystructure of my mods accordingly.
And they are loaded as intended.

But this metadata.yaml file doesn't seem to work.

Code: [Select]
name: "Hardmode Expansion Mod"
version: 0.7 Beta
description: "This mod intends to make the gameplay more challenging, featuring a couple of new races, new Weapons for Xcom and the Aliens. You will encounter bigger Alien Crews on all Missions."
author: "hellrazor"
master: "xcom1"

It exists in the Mods main directory.

PLEASE change your mky002!
The file is called metadata.yml not *.yaml. :)
Title: Re: Upgrading to the nightly
Post by: Meridian on May 07, 2015, 03:14:42 pm
It was renamed to "metadata.yml" in the meantime.
Title: Re: Upgrading to the nightly
Post by: hellrazor on May 07, 2015, 03:15:47 pm
It was renamed to "metadata.yml" in the meantime.

Well i just checked the defaults ruleset they have so it occured to me also.
Title: Re: Upgrading to the nightly
Post by: Ishmaeel on May 07, 2015, 03:20:44 pm
sure, stress testing would be appreciated at this point

Well, it didn't go as well as I hoped it would. As of now, UFO and TFTD subfolders are hardcoded, correct?
https://github.com/SupSuper/OpenXcom/blob/ff7f14193030922124bdc981e4b98de0d3ab302f/src/Engine/Options.cpp#L290 (https://github.com/SupSuper/OpenXcom/blob/ff7f14193030922124bdc981e4b98de0d3ab302f/src/Engine/Options.cpp#L290)

I was hoping to override those with the -data switch or at least by editing the respective metadata.yml files to be able to use an in-place Steam-based installation.

Oh well, I'll just duplicate the XCOM folder as UFO for now.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 07, 2015, 05:08:42 pm
I'll update the second post documentation according to recent commits.

Warboy updated the yaml->yml change, and I just added in some clarification around sprite limits:
Quote
the sprite limit is now *per mod*, not *per ruleset*.  this means that you can split your ruleset into multiple files and sprites declared in one can be used in the others.  The sprite limit is still 1000 for regular mods, but is effectively infinite for masters.  Masters can use as many sprite numbers as they want.  Any mods that declare a master, though, must only use numbers higher than what the master uses to ensure no collisions.  So, for example, if a master uses ids for sprites up to 4350, maybe all mods for that master could start their ids at 6000 or so.  Even if there is id overlap *between mods*, the engine will keep them straight and each mod will refer correctly to its own sprites.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 07, 2015, 05:35:32 pm
Well, it didn't go as well as I hoped it would. As of now, UFO and TFTD subfolders are hardcoded, correct?

I was hoping to override those with the -data switch or at least by editing the respective metadata.yml files to be able to use an in-place Steam-based installation.
Well, the *detection* of UFO and TFTD is hard coded, but the actual loading of data files is not.  If you want to get around having a copy of the data, you can point the -data parameter to one level up from the steam XCOM folder, add a file named TERRAIN/UFO1.PCK to the UFO directory (it can be an empty file, it just has to exist), and change the loadResources line in xcom1's metadata.yml file to read XCOM instead of UFO.  I haven't tested this, but it should work.
Title: Re: Upgrading to the nightly
Post by: Ishmaeel on May 07, 2015, 05:52:00 pm
...add a file named TERRAIN/UFO1.PCK to the UFO directory (it can be an empty file, it just has to exist), and change the loadResources line in xcom1's metadata.yml file to read XCOM instead of UFO.

I tested it. It does work. Thanks.

Edit: Didn't even have to specify the -data argument. Creating UFO directory right within Steam game folder was sufficient.

This hadn't even occurred to me. I thought there would be a more involved process after the initial detection because the error page complains about GEODATA/PALETTES.DAT being not found. :o
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 07, 2015, 08:22:17 pm
Since we have to re-install OpenXcom, reinstall mods, and re-enable said mods, will this force a restart of campaigns? I'm hesitant to upgrade right this second due to fear of having to start all over again.
Title: Re: Upgrading to the nightly
Post by: arrakis69ct on May 07, 2015, 08:38:51 pm
THX FOR THE WORK.

waiting the tftd thx
Title: Re: Upgrading to the nightly
Post by: Bloax on May 07, 2015, 08:39:30 pm
Since we have to re-install OpenXcom, reinstall mods, and re-enable said mods, will this force a restart of campaigns? I'm hesitant to upgrade right this second due to fear of having to start all over again.
Step by Step Guide for Upgrading:
  • Copy your original UFO data files somewhere safe
  • Uninstall OpenXcom and remove any files left behind in the "data" directory
  • Install OpenXcom
  • Copy your original UFO data files into the UFO directory
  • Start OpenXcom to make sure it loads ok
  • Extract any mods you want to install into the "mods" directory, beneath where your savegames are located.  Ensure each mod is extracted into a separate directory under the mods directory.  See the example below.
  • Start OpenXcom, go to Options -> Mods, and enable mods you want to use.
  • Your old savegames will work just fine as long as the same mods you had loaded before are still loaded.
Keep a backup of the old openxcom folder in case you mess it up.
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 07, 2015, 08:49:51 pm
Oh. Shit.  :-[ My am dumb
Title: Re: Upgrading to the nightly
Post by: XOps on May 07, 2015, 09:23:46 pm
Awesome. This will help greatly with keeping all of that terrain stuff in order. Question. Is there a way to enable or disable individual rulesets within a mod like say if a mod had an alternate ruleset? It appears as though that selecting a mod by default enables all rulesets at once within the mod folder.
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on May 07, 2015, 09:54:28 pm
It appears as though that selecting a mod by default enables all rulesets at once within the mod folder.

correct, but you CAN have mods reference data in other folders, so you could make it "modular" in that sense
Title: Re: Upgrading to the nightly
Post by: Hobbes on May 07, 2015, 10:08:56 pm
Why? Why? Why? We're doomed! We're all doomed! Great work! :D

I'll see if I can upgrade today the Terrain Pack to the newest nightly (which should also help distracting me from home improvement hell).

Awesome. This will help greatly with keeping all of that terrain stuff in order.

One thing I noticed is that there has been also a change to make alien itemLevels less restrictive, so ideally modders won't have to create their own Terrain Pack rulesets because of different itemlevels.
Title: Re: Upgrading to the nightly
Post by: Angelus_EV on May 07, 2015, 10:16:34 pm
at the translation site Spanish (latam) is 100% translated but Latest Nightly have untranslated strings!
on the other hand.... awesome job!
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on May 07, 2015, 10:25:35 pm
the transifex issues will be sorted out over the weekend with any luck
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 08, 2015, 12:38:45 am
Good God thank you so much for the step-by-step process on how to get this mofo running. My face probably looked like I was pushing out a growler waiting to see if I got it working correctly. Just a quick question, the default, un-modded music seems different now (like there's more to it). Did something change with that or am I just losing my mind?
Title: Re: Upgrading to the nightly
Post by: myk002 on May 08, 2015, 01:01:11 am
Did something change with that or am I just losing my mind?
It's always /possible/ that the code changes are unintentionally causing different files to get picked up, but I still suspect the latter option...
If anyone else notices anything, please speak up.
Title: Re: Upgrading to the nightly
Post by: DoxaLogos (JG) on May 08, 2015, 02:23:43 am
This is way cool.  Warboy now needs to change his personal text to "Breaker of Mods". :)
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 08, 2015, 03:24:12 am
Ok I double-checked with my old version and the audio is DEFINITELY different. However I think I found out what happened; My old default setting was adlib this new version is set to midi format. No matter what though, it continually reverts to midi format even if I select a different output setting.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 08, 2015, 03:52:40 am
Ok, I found something in the code that *might* be relevant.  Would you be able to build the branch to test to see if what I changed fixes things for you?

branch: https://github.com/myk002/OpenXcom/tree/adlib_fix
commit: https://github.com/myk002/OpenXcom/commit/62347d2428eae8b751e2afbb15d762126c4a3d38  (in case you want to apply it yourself)
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 08, 2015, 03:57:31 am
I'm not quite sure what to do. I've got zero experience with coding. Maybe I should just wait for a nightly.

Title: Re: Upgrading to the nightly
Post by: darkestaxe on May 08, 2015, 04:14:20 am
sure, stress testing would be appreciated at this point

I was able to run 3 OXC processes at once with the new build, with max SSAA/SSAO + Ultra settings, and kept a steady 60FPS on the geoscape. However, my heat-sink fan turned on with the third simultaneous process so I'm not sure how long that would have lasted, but then I am using a crummy old netbook.

Ya I'm a smartass.

This is way cool.  Warboy now needs to change his personal text to "Breaker of Mods". :)

Ya WTH Warboy, why are my saves still working!?! Seriously though, I'm loving this 'break everything all at once' approach.

And really nice instructions from myk002. Big THX all around guys and great work!

I'm not quite sure what to do. I've got zero experience with coding. Maybe I should just wait for a nightly.

Can you give him the .dll?
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 08, 2015, 04:20:03 am
You mean attach it to my post? Sure I can do that. Which .DLL is it? I was effing with the sound option, the one that says 1.4 or 1.0 and setting it to 1.0 gave me an error that SAMPLE.CAT could not be found. I fixed it but won't be fiddling with that anytime soon.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 08, 2015, 04:31:56 am
I think he meant can I give you the (built) dll.  Sure -- it will take a few hours, the windows VM I have for development is *not* fast.  If you want an OSX or a Linux build, I can get it to you much faster.
Title: Re: Upgrading to the nightly
Post by: darkestaxe on May 08, 2015, 04:40:19 am
I think he meant can I give you the (built) dll.  Sure -- it will take a few hours, the windows VM I have for development is *not* fast.  If you want an OSX or a Linux build, I can get it to you much faster.

Ya, I should have been more clear. If Myk gives Pheonix the fixed Adlib file as a DLL instead of C++ code... pheonix could use it.
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 08, 2015, 04:48:31 am
Oh sure I can try that. Take your time on it it's not like OMG I NEEDZ IT NOW kind of thing.
Title: Re: Upgrading to the nightly
Post by: kkmic on May 08, 2015, 09:48:03 am
@myk002: How do you translate the mod description? The one from the metadata.yml file...
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 08, 2015, 10:07:51 am
Well, what should I say? Works like charm! And the changes for my compilation-page aren't that extensive. Thanks for structuring it a bit more. Especially that mod-folder in the user-space is nice.
Title: Re: Upgrading to the nightly
Post by: niculinux on May 08, 2015, 01:16:54 pm
Super! MAy we get some sort of updgradin video, for eg. on the openxcom official youtube channel (https://www.youtube.com/user/openxcom)? Would be "ultimate"!
Title: Re: Upgrading to the nightly
Post by: yrizoud on May 08, 2015, 05:26:45 pm
Sorry if I'm missing something, but I can only find english language (gb and us) in the nightly zip and in github's zip
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on May 08, 2015, 05:33:31 pm
translation stuff should (hopefully) be sorted out over the weekend, in the meantime, just use https://openxcom.org/translations/latest.zip (https://openxcom.org/translations/latest.zip)
Title: Re: Upgrading to the nightly
Post by: myk002 on May 08, 2015, 07:00:56 pm
@myk002: How do you translate the mod description? The one from the metadata.yml file...
Currently, it is not translatable.
@Warboy and @SupSuper - is this included in the translation issues you plan on addressing this this weekend?
Title: Re: Upgrading to the nightly
Post by: Ishmaeel on May 08, 2015, 11:27:16 pm
Since the translation issue has come up, I probably should bring up an issue I've been holding on.

The mod selection screen has a performance issue with untranslated strings. Basically, the screen tries to update the mouseover description on every mouse frame (even if the currently hovered item hasn't changed) and if no translation exists in the currently selected language, this leads to horrible mouse lag. I suspect the delay is caused by the debug messages generated when a string lookup fails.

This issue is also present in the advanced options screen where a similar hover behavior is present. It may just not be as pronounced as the mod screen because the translations are not missing "by default" :P

I was able to fix this on my end by updating the mouseover description only when the current item has changed. I could put up a PR for it, but I didn't trust my C++ or familiarity with the OXC code. Would you like a bug report instead?
Title: Re: Upgrading to the nightly
Post by: Alex_D on May 09, 2015, 08:46:14 am
This should be interesting.  :)

Let me see if my poor brain understood the installation instructions.

On step two: I delete everything except for the "user" folder. I installed OXC ver1.0 from the zip file and I have my save games on a subfolder alongside with "data". This step preserves essentially only the save games and the configurations.

My understanding is any .dll files of version 1.0 are not longer needed as the compiled nightly has an exe that is self contained in that regard.

On step three: By "Install OpenXcom" it is meant to say install the contents of the latest nightly, which I see they have obviously the new folder structure. The files of version 1.0 are superseded and not need to be installed again before the installation of the nightly, as it used to be the case.

On step four: The original UFO folders are to be copied on the UFO folder. The universal patch files must be also copied along as well.

On step six: The mods folder that I should create would be "\OpenXcom\user\mods". And there is where I dump the mods to use, each on its own folder.

Future nightlies, for a clean install, would require the "common", "standard" folders, and the "openxcom.exe", "CHANGELOG.txt", and "LICENSE.txt" files, to be deleted. Or the other way around, "UFO", "TFTD", and "user" folders to be preserved while everything else is deleted prior to the new nightly installation.

Did I get right? I ran a quick battle and it appeared fine.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 09, 2015, 11:22:40 am
I suspect the delay is caused by the debug messages generated when a string lookup fails.
Updating the code to only generate the tooltip if it is going to change would solve the problem for the mods screen, but there are other places in the code that would run into the same issue.  I tried to solve the problem in a slightly more generic way -- by ensuring the output warning is only generated once per string id.  https://github.com/SupSuper/OpenXcom/pull/1011

@Alex_D: right on all counts.
yeah, openxcom 1.0 is not required at all if you are installing the nightly version.
you are correct in applying the universal patch files over the files in the UFO directory.  This is probably the simplest method.  You could also put them in a subdirectory of the mods folder and overwrite them like a mod would, but unless you really need to keep the original UFO files pristine, there's no reason to do that.
Title: Re: Upgrading to the nightly
Post by: pilot00 on May 09, 2015, 07:40:12 pm
Ok I double-checked with my old version and the audio is DEFINITELY different. However I think I found out what happened; My old default setting was adlib this new version is set to midi format. No matter what though, it continually reverts to midi format even if I select a different output setting.

I have a sound problem too with the new version. It plays normal for several minutes, then it abruptly starts to stop,start,stop (stutter? I dont know the right word) and then it stops altogether till I restart the game.

Its a win7 machine.

EDIT: And now apparently its mute even when I restart  :o
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 09, 2015, 08:05:01 pm
I sent out a PM but I'll put it here too: my audio is still forced into MIDI no matter what I select. I even set it back to ad-lib on my back-up of OpenXcom and it still reverts back to MIDI on the new versions.

Windows 7 64-bit here. I have an 8.1 laptop that I haven't yet tested too.
Title: Re: Upgrading to the nightly
Post by: Alex_D on May 09, 2015, 10:48:08 pm
I just found that if I remove a previously selected master mod by deleting its folder, the game appears to default to xcom2, and thus crashes. The option.cfg file needs editing to return to xcom1 as master.

Also, in the past, a mod could be used by other mods. Now, if a master is selected, only the mods that have it declared as master can be activated in the mods option menu. So how do I make a mod to be usable by two masters (e.g.: xcom1, and Piratez)? Do I need to create two copies (two folders) of the mod and fiddle with the metadata?
Title: Re: Upgrading to the nightly
Post by: darkestaxe on May 10, 2015, 01:18:13 am
In case it matters to anyone, I'm also on Win 7 x64 and I have no problems switching between Adlib, MIDI, and OGG.

Version:2015-05-08 0634
Title: Re: Upgrading to the nightly
Post by: myk002 on May 10, 2015, 02:29:47 am
@pilot00 - are you having a problem similar to Phoenix7786?  is it picking up a different /type/ of music than it did before?
Title: Re: Upgrading to the nightly
Post by: myk002 on May 10, 2015, 04:32:19 am
I just found that if I remove a previously selected master mod by deleting its folder, the game appears to default to xcom2, and thus crashes.
do you happen to have the TFTD data installed?

Quote
Now, if a master is selected, only the mods that have it declared as master can be activated in the mods option menu. So how do I make a mod to be usable by two masters (e.g.: xcom1, and Piratez)? Do I need to create two copies (two folders) of the mod and fiddle with the metadata?
I actually originally had this feature in the original code, but then I wondered, exactly what kind of mod would apply to more than one master?  So I took it out to simplify things.  I did specify that if a mod is /generally/ applicable to all masters, such as StatStrings, it could specify "*" as the master and be always loadable.  What kind of mod did you have in mind that could apply, unchanged, to both Piratez and vanilla xcom1?
Title: Re: Upgrading to the nightly
Post by: Alex_D on May 10, 2015, 06:02:24 am
do you happen to have the TFTD data installed?
Yes. I populated the UFO and TFTD folders as well.

So, if I infer correctly, having anything in TFTD but the README.txt that comes with the nightly would cause OXC to believe that TFTD is active?

I actually originally had this feature in the original code, but then I wondered, exactly what kind of mod would apply to more than one master?  So I took it out to simplify things.  I did specify that if a mod is /generally/ applicable to all masters, such as StatStrings, it could specify "*" as the master and be always loadable.  What kind of mod did you have in mind that could apply, unchanged, to both Piratez and vanilla xcom1?

I was thinking on Hobbes' terrain pack mod. On the other hand Piratez is moving to OXC Extended and the current version 0.9j may be the latest on "vanilla". And Piratez has extensive use of Hobbes' maps albeit (and I can be wrong) without the need to activate the ruleset, just the maps overwrite Piratez's maps for latest bug fixing. Hence copying manually the Terrain pack files into Piratez may fix the problem without declaring the Terrain pack as a master="*".
Title: Re: Upgrading to the nightly
Post by: myk002 on May 10, 2015, 06:32:05 am
we'll have to see what mods actually need.  if it looks like declaring multiple specific masters is needed, we can always code it back in.

regarding tftd getting autoselected when you remove the active master mod dir: the detection routine has been fixed (see https://github.com/SupSuper/OpenXcom/pull/1012 ) so that shouldn't happen anymore.  tftd won't be listed as a game type until warboy releases the ruleset.
Title: Re: Upgrading to the nightly
Post by: Phoenix7786 on May 10, 2015, 06:52:12 am
Sound issue is fixed. For some reason my raw UFO Defense SOUND folder is missing a bunch of audio files. I extracted the missing files from my back-up of OpenXcom and it worked like a charm.
Title: Re: Upgrading to the nightly
Post by: pilot00 on May 11, 2015, 01:17:37 pm
@pilot00 - are you having a problem similar to Phoenix7786?  is it picking up a different /type/ of music than it did before?

Not exactly, my problem was stuturing. I managed to fix my issue by an odd way that I cant guarentee it will work on you guys. I just found out the sound files and replaced them from a friends copy of the game (gold edition).

What I noticed though, is that it sometimes continues to play the battlescape track after finishing a mission, till the skyranger returns to base. But that might be an effect of the process I did, so dont hold on to it.

EDIT: ANd now I see that phoenix did the same LOL.

What I have a problem though is this: I cant move the files of the game into another folder if I want. When it creates the save/mod files, it always creates them on a specific path. If I move the folder it cant find the mods/saves nor it creates a new path. The only way to use them is to return it to its exact same spot.This is frustrating for two reasons: I am doing the copy/paste jobs on my desktop before I move them to the right file, so it kinda forces me to do it in a specific way.

But thats not a problem, the problem is that I want to have two different instalations of the game, how do I go about changing the path/files where it will create and draw the mods and saves from?
Title: Re: Upgrading to the nightly
Post by: redv on May 11, 2015, 03:54:28 pm
  • the list of mods is actually a list of mods, not a list of rulesets like before.  if a mod has multiple rulesets, you only need to enable one item

The AWACS mod has two rulesets:
1. AWACS.rul  only adds AWACS, Hawkeye and Darkstar aircrafts to current game.
2. AWACS.scenario.rul  adds aircrafts, deletes conventional radars and changes the starting base layout. Therefore this ruleset used for brand new game.
https://www.openxcom.com/mod/awacs-aircraft

Both rulesets uses the same resources, but should be used only one of them, not both.

@myk002
How to enable/disable these rulesets separately in the new "mods" menu?
Title: Re: Upgrading to the nightly
Post by: myk002 on May 11, 2015, 07:51:16 pm
How to enable/disable these rulesets separately in the new "mods" menu?
This is a question that I've given some thought to, and struggled with while designing the new system.  So settle in, this will be a long, technical post.

Within the system as it now stands, you'd handle this by having all resources and a "baseline" mod in one top-level directory, and ruleset changes for the "alternate" version in a second top-level directory.  Users would have either one mod active or both, but never the second without the first.  We will be able to enforce this with requires: tags in the metadata.yml files soon.

The only caveat is that sprite: references in any rules MUST refer to sprites that are defined by either the same rul file or in a rul file in the same directory.  From looking at the AWACS mod, it looks like it won't be a problem (it's ok if both are enabled since AWACS.scenario is a strict superset of AWACS), but in general, a mod can't refer to a sprite offset that is defined by a different mod due to the "index anti-collision" algorithm that is in place.

You'd package your mod like this:
Code: [Select]
AWACS/metadata.yml
AWACS/AWACS.readme.txt
AWACS/Resources/AWACS/*.*
AWACS/Ruleset/AWACS.rul
AWACS.scenario/metadata.yml
AWACS.scenario/Ruleset/AWACS.scenario.rul

and the user would have two directories in the mods dir: AWACS and AWACS.scenario.  They would either enable just AWACS or both AWACS and AWACS.scenario.

The guiding principle for the original design was "make it as simple as possible for the user", and I'd really like to keep it that way.  Of course, the second principle was "make it as simple as possible for mod creators" : )  There were a couple alternate approaches that I considered, but either I or Warboy rejected on the grounds that it was too complicated (but, if they become necessary, we can revisit them):


My hope is that the existing, simple system is enough, but the conversation for improvement is always open.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 11, 2015, 07:58:39 pm
What I have a problem though is this: I cant move the files of the game into another folder if I want. When it creates the save/mod files, it always creates them on a specific path. If I move the folder it cant find the mods/saves nor it creates a new path. The only way to use them is to return it to its exact same spot.This is frustrating for two reasons: I am doing the copy/paste jobs on my desktop before I move them to the right file, so it kinda forces me to do it in a specific way.

But thats not a problem, the problem is that I want to have two different instalations of the game, how do I go about changing the path/files where it will create and draw the mods and saves from?
having two different installations for the game is not hard.  just use -user, -config, and -data parameters when running the game to explicitly set which folders to use.  This is documented in the top-level README.md file for OpenXcom.  You can read it at https://github.com/SupSuper/OpenXcom if you just scroll down past the file list.  ah, here's a direct link to that spot on the page: https://github.com/SupSuper/OpenXcom#directory-locations
Title: Re: Upgrading to the nightly
Post by: pilot00 on May 12, 2015, 12:37:08 am
Did try to edit the config folder but no results. It kept ignoring me  :'(
Title: Re: Upgrading to the nightly
Post by: myk002 on May 12, 2015, 01:54:51 am
Did try to edit the config folder but no results. It kept ignoring me  :'(
I need to add this to the README.md -- to set the config folder, the commandline param is -cfg.  that's terribly confusing.  actually, I'll see if I can just add -config as a valid variant.  for now, though, use -cfg
Title: Re: Upgrading to the nightly
Post by: hellrazor on May 12, 2015, 11:30:38 am
On another regard.
How does the explicit new mod structure look?

At the moment i use the following structure:

Code: [Select]
Hardmode_Expansion/MAPS
Hardmode_Expansion/ROUTES
Hardmode_Expansion/TERRAIN
Hardmode_Expansion/Ruleset/*.rul (several files, split up in sections like xcom1 ruleset -> all items are in items.rul for example)
Hardmode_Expansion/Resources/Hardmode_ExpansionBigOb/
Hardmode_Expansion/Resources/Hardmode_ExpansionHandOb/
Hardmode_Expansion/Resources/Hardmode_ExpansionFloorOb/
Hardmode_Expansion/Resources/Hardmode_Expansion/Geoscape/
Hardmode_Expansion/Resources/Hardmode_Expansion/Inventory/
Hardmode_Expansion/Resources/Hardmode_Expansion/Sounds/
Hardmode_Expansion/Resources/Hardmode_Expansion/SpriteSheet/
Hardmode_Expansion/Resources/Hardmode_Expansion/UfoPaedia/
Hardmode_Expansion/CHANGELOG.TXT
Hardmode_Expansion/INSTALL.TXT
Hardmode_Expansion/metadata.yml
Hardmode_Expansion/CREDITS.TXT
Hardmode_Expansion/README.TXT

What do i need to change to be completly on the new System?

How can i define multiple Submods?
It was possbiel before to have several ruleset files and active them one by one.
I presume i have to set them in seperate directories now.

Also some more questions:

- How can i access files directly from other Mods MAPS, ROUTES or TERRAIN directories or even sprites?
- How are spritesnumbers handled now, before, every loaded ruleset had its own range defined.
Title: Re: Upgrading to the nightly
Post by: pilot00 on May 12, 2015, 04:10:56 pm
I need to add this to the README.md -- to set the config folder, the commandline param is -cfg.  that's terribly confusing.  actually, I'll see if I can just add -config as a valid variant.  for now, though, use -cfg

You got me a little bit confused :). Doesnt editing the config file it spawns at the mod/save file location what you meant? Because thats what I did but so far no candy.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 12, 2015, 07:19:07 pm
For the self-builders here: Is there any way, to automate the cc-make-command that way, that the CMAKE_BUILD_TYPE-variable is set to »Release« and configuration and generation is being started right away? Would help automated compilation quite a lot.
Title: Upgraded to nightly and game can no longer find my data (LINUX - UBUNTU)
Post by: abitlost on May 12, 2015, 11:53:39 pm
I've been using OpenXcom (Thank you for making this BTW, fantastic!!) but seeing the announcement I thought I'd make the jump to the new nightly with changed data folder locations. Installed all the dependencies, followed the "Compiling with Make" instructions on the ufopaedia wiki, compilation went fine, etc. and copied the data over for both UFO and TFTD (my version is from the old CDs, not the Steam release) but I keep getting an error "GEODATA/PALETTES.DAT not found" when I run the binary it created in the OpenXcom/bin folder.

I've tried a few different things. The log file in the bin folder showed that it created the mods folder in my ~/.local/share/openxcom/mods That's where I first tried putting the UFO/TFTD folders.
There are also folders "common", "standard", "TFTD" and "UFO" in that same bin folder as the executable. I tried putting the data there, same problem. I tried running it with the -data option, pointing it directly to ~/.local/share/openxcom/ same issue.

I read the log in the ~/.local/share/openxcom folder. The first line after "Data search is:" shows ~/.local/share/openxcom, where I first put the data. I read the data locations article at  https://github.com/SupSuper/OpenXcom#directory-locations and that tells me the data should be in "$HOME/.local/share/openxcom" (as I understand it $XDG_DATA_HOME is not defined in Ubuntu) and it is indeed in there so feeling a little lost.

I've had a look over the forums and saw someone mention a similar problem on earlier versions (https://openxcom.org/forum/index.php/topic,3485.0.html). They found running as root solved it for them, maybe suggesting a permissions issue on folders,  but didn't make any difference to me. Case sensitivity was mentioned. Never a problem on any previous released milestones but tried it anyway, no difference.

Any help appreciated.
Title: Re: Upgraded to nightly and game can no longer find my data (LINUX - UBUNTU)
Post by: myk002 on May 13, 2015, 02:58:08 am
but I keep getting an error "GEODATA/PALETTES.DAT not found" when I run the binary it created in the OpenXcom/bin folder.

welcome to the forums : )  Could you post the full contents of your openxcom.log file?  You can PM it to me if you don't want to post it publicly.
Title: Re: Upgrading to the nightly
Post by: myk002 on May 13, 2015, 04:26:46 am
What do i need to change to be completly on the new System?
that structure looks good.  You could optionally move the ruleset files to the top level directory, but it's not required.

Quote
How can i define multiple Submods?
redv just asked a similar question.  see: https://openxcom.org/forum/index.php/topic,3617.msg44622.html#msg44622

Quote
- How can i access files directly from other Mods MAPS, ROUTES or TERRAIN directories or even sprites?
- How are spritesnumbers handled now, before, every loaded ruleset had its own range defined.
files that you access by name, like those in MAPS, ROUTES, and TERRAIN are just accessed by relative path like before.  you can access another mod's sprites, but you have to declare the extrasprites in one of your own ruleset files.  I touched on the sprite situation in the message linked above too.

You got me a little bit confused :). Doesnt editing the config file it spawns at the mod/save file location what you meant? Because thats what I did but so far no candy.
I might have misunderstood you -- to have a completely independent openxcom installation, you need to specify unique config, user, and data dirs.  like this:
Code: [Select]
/path/to/openxcom -cfg "/path/to/unique/config/dir" -user "/path/to/unique/user/dir" -data "/path/to/unique/data/dir"
I had thought you had tried this, but used -config instead of -cfg.  The documentation I wrote is not clear that the current code accepts -cfg but not -config.  I'm going to submit a pull request so that both forms are accepted.

Is there any way, to automate the cc-make-command that way, that the CMAKE_BUILD_TYPE-variable is set to »Release« and configuration and generation is being started right away?
from a fresh git checkout, run:
Code: [Select]
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j10
so this way, you don't need to run ccmake at all, just cmake.  you can set other vars on the initial call to cmake too.  This is the one I use to get all set up on OSX:
Code: [Select]
CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/bin/openxcom.inst ..
Title: Re: Upgrading to the nightly
Post by: darkestaxe on May 13, 2015, 05:03:56 am
Here's a few ideas for implementing mod options without making it too complicated.


Mods could have an options.xml file (not required) that would look something like this:
Code: [Select]
/* Only one string */
<option "Include original grenade (don't replace)">Rulesets\KeepGrenade.rul</option>

/* or multi-language */
<option STR_KEEP_GRENADE>Rulesets\KeepGrenade.rul</option>

This could later be expanded to allow for things like:
Code: [Select]
<optiongroup "Hyperwave Decoder Options:">
  <option "Hyperwave Decoder doesn't add any detection">NoDetect.rul</option>
  <option "Hyperwave Decoder range is unlimited but only has 10% detection">InfiniteRange.rul</option>
  <option "Same as Vanilla"></option>
</optiongroup>

I would be in favor of the xml approach because it would be a lot easier for modders then YAML. But since OXC already uses so much YAML, another approach would be to incorporate the options into Metadata.yml. The descriptions would only be needed if there's no corresponding entries in a language file.
Code: [Select]
Options:
  -  Name: STR_KEEP_GRENADE
     Description: "Include original grenade (don't replace)"
     Ruleset: KeepGrenade.rul
OptionGroups:
  -  Name: STR_HYPERWAVE_OPTIONS
     Description: "Hyperwave Decoder Options:"
     -  Name: STR_NO_DETECT
        Description: "Hyperwave Decoder doesn't add any detection"
        Ruleset: NoDetect.rul
     -  Name: STR_INFINITE_RANGE
        Description: "Hyperwave Decoder range is unlimited but only has 10% detection"
        Ruleset: InfiniteRange.rul
     -  Name: VANILLA
        Description: "Same as Vanilla"
        Ruleset: []

My final idea would be to allow options to be externalized from metadata.yml to options.rul or options.xml, and allow the exact same code as above, but also allow things like:
Code: [Select]
facilities:
  - type: STR_HYPER_WAVE_DECODER
    hyper: STR_HYPER
    radarRange: STR_RANGE
    radarChance: STR_CHANCE
options:
  -  name: STR_HYPER
     description: "Extra Hyperwave-info in radar screens:"
     type: bool
     default: true
  -  name: STR_RANGE
     description: "Maximum Detection Range (1200 for whole globe):"
     type: int
     default: 12000
  -  name: STR_CHANCE
     description: "Percent chance of detecting an undetected UFO each scan:"
     type: int
     range:
        [0, 100]
     default: 10
or with an options.xml
Code: [Select]
<rule>
facilities:
  - type: STR_HYPER_WAVE_DECODER
    hyper: <bool STR_HYPER>true</bool>
    radarRange: <int STR_RANGE>1200</int>
    radarChance: <int STR_CHANCE min=0 max=100>10</int>
</rule>

Keep in mind that with each of these suggestions modders are still able to have optional rulesets just like before using the grenade example. As a matter of making things easier for modders I personally would greatly prefer XML because it's not evil. XML does not sneak into a persons room at night to count and judge all your whitespace chars like some kind of anal-retentive nazi. XML also doesn't take sides in the bloody line-break war. XML hasn't chosen /n as its glorious standard and doesn't execute impure (/r or /n/r) line-breaks in the name of UNIX supremacy.
Title: Re: Upgraded to nightly and game can no longer find my data (LINUX - UBUNTU)
Post by: AndO3131 on May 13, 2015, 09:17:28 am
Quote
I read the log in the ~/.local/share/openxcom folder. The first line after "Data search is:" shows ~/.local/share/openxcom, where I first put the data.
Recently data structure has changed https://openxcom.org/forum/index.php/topic,3617.0.html (https://openxcom.org/forum/index.php/topic,3617.0.html). Try putting your data in ~/.local/share/openxcom/UFO folder. If this doesn't help, try to delete ~/.config/openxcom/options.cfg file.
Title: Re: Upgraded to nightly and game can no longer find my data (LINUX - UBUNTU)
Post by: abitlost on May 13, 2015, 09:48:53 am
Thanks for the suggestions. My data is in ~/.local/share/openxcom/UFO but it didn't find it :( Tried deleting the ~/.config/openxcom/options.cfg and still the same.

To make sure I wasn't going crazy I tried reinstalling 1.0 through the Ubuntu repo that was set up, put the data in ~/.local/share/openxcom/data and ran first time. I then uninstalled it but using synaptic this time (mark for complete removal) to see if dug any other lingering stuff from 1.0 out. When I try my compiled version it's still throwing the same error so wondering if it's something I've done wrong there. :(

myk002, thank you :)  Here's my full openxcom.log:


Code: [Select]
[13-05-2015 07:47:06] [INFO] Data folder is:
[13-05-2015 07:47:06] [INFO] Data search is:
[13-05-2015 07:47:06] [INFO] - /home/mike/.local/share/openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/share/xubuntu/openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/share/xfce4/openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/local/sharehttps://openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/sharehttps://openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/share/openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/local/share/openxcom/
[13-05-2015 07:47:06] [INFO] - /usr/share/openxcom/
[13-05-2015 07:47:06] [INFO] - ./
[13-05-2015 07:47:06] [INFO] User folder is: /home/mike/.local/share/openxcom/
[13-05-2015 07:47:06] [INFO] Config folder is: /home/mike/.config/openxcom/
[13-05-2015 07:47:06] [INFO] Options loaded successfully.
[13-05-2015 07:47:06] [INFO] Scanning standard mods in 'standard'...
[13-05-2015 07:47:06] [INFO] - standard/Aliens_Pick_Up_Weapons
[13-05-2015 07:47:06] [INFO] - standard/Limit_Craft_Item_Capacities
[13-05-2015 07:47:06] [INFO] - standard/PSX_Static_Cydonia_Map
[13-05-2015 07:47:06] [INFO] - standard/UFOextender_Gun_Melee
[13-05-2015 07:47:06] [INFO] - standard/UFOextender_Psionic_Line_Of_Fire
[13-05-2015 07:47:06] [INFO] - standard/UFOextender_Starting_Avalanches
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Always_Daytime
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Always_Nighttime
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Fighter_Transports
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_High_Explosive_Damage
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Improved_Ground_Tanks
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Improved_Heavy_Laser
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_No_Psionics
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Pistol_Auto_Shot
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Skyranger_Weapon_Slot
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Starting_Defensive_Base
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Starting_Defensive_Improved_Base
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Starting_Improved_Base
[13-05-2015 07:47:06] [INFO] - standard/XcomUtil_Statstrings
[13-05-2015 07:47:06] [INFO] - standard/xcom1
[13-05-2015 07:47:06] [INFO] - standard/xcom2
[13-05-2015 07:47:06] [INFO] Scanning user mods in '/home/mike/.local/share/openxcom/mods'...
[13-05-2015 07:47:06] [ERROR] no mod masters available
[13-05-2015 07:47:06] [INFO] Mapping resource files...
[13-05-2015 07:47:06] [ERROR] cannot determine current active master
[13-05-2015 07:47:06] [INFO]   mapping resources in: common
[13-05-2015 07:47:06] [INFO] SDL initialized successfully.
[13-05-2015 07:47:06] [INFO] SDL_mixer initialized successfully.
[13-05-2015 07:47:06] [INFO] Attempting to set display to 640x400x8...
[13-05-2015 07:47:06] [INFO] Display set to 640x400x8.
[13-05-2015 07:47:06] [INFO] Loading rulesets...
[13-05-2015 07:47:06] [INFO] Rulesets loaded successfully.
[13-05-2015 07:47:06] [INFO] Loading resources...
[13-05-2015 07:47:06] [INFO] requested file not found: GEODATA/PALETTES.DAT
[13-05-2015 07:47:06] [ERROR] GEODATA/PALETTES.DAT not found
 
Title: Re: Upgrading to the nightly
Post by: winterheart on May 13, 2015, 10:52:14 am
For the self-builders here: Is there any way, to automate the cc-make-command that way, that the CMAKE_BUILD_TYPE-variable is set to »Release« and configuration and generation is being started right away? Would help automated compilation quite a lot.

cmake -DCMAKE_BUILD_TYPE=Release <where your sources are>
make
Title: Re: Upgrading to the nightly
Post by: AndO3131 on May 13, 2015, 01:28:46 pm
Are there any folders in PATH_TO_YOUR_COMPILED_BINARY_FILE ? I run ./openxcom from there and put data files in PATH_TO_YOUR_COMPILED_BINARY_FILE/UFO directory.
Title: Re: Upgraded to nightly and game can no longer find my data (LINUX - UBUNTU)
Post by: myk002 on May 13, 2015, 06:01:20 pm
I followed up with abitlost over PM.  There is a bug in the "do you have UFO data installed?" routine that applies to his particular setup.  I'll submit a fix so no one else runs into this.

edit: submitted https://github.com/SupSuper/OpenXcom/pull/1019 to resolve case sensitivity issues when detecting whether the game data is installed and for allowing UFO and TFTD folders in the user dir.
Title: Re: Upgrading to the nightly
Post by: abitlost on May 13, 2015, 08:22:59 pm
Thanks again for replying so quickly myk002, your fix worked perfectly! You're a star!!  ;D
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 13, 2015, 11:01:59 pm
Code: [Select]
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j10
so this way, you don't need to run ccmake at all, just cmake.  you can set other vars on the initial call to cmake too.  This is the one I use to get all set up on OSX:
Code: [Select]
CC=/usr/bin/clang CXX=/usr/bin/clang++ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/bin/openxcom.inst ..
And when not entering a value after
Code: [Select]
make -j it runs awesomely fast. That kicks ass. Thanks for the hints (also to winterheart). Now I only have to find a way, to automatically generate something for the version-string, locally, and I'll be happy. =)
Title: Re: Upgrading to the nightly
Post by: myk002 on May 14, 2015, 02:37:34 am
cmake will pick up the git revision so that should be covered too
Title: Re: Upgrading to the nightly
Post by: Arthanor on May 14, 2015, 02:48:26 am
Interesting.. make -j uses however many jobs you can, whereas -j10 would limit it to 10 jobs. I would have expected that 10 jobs would be high enough. I'll have to try it since taking forever is one of the main reasons why I don't update more often..
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 14, 2015, 11:50:55 am
cmake will pick up the git revision so that should be covered too
Apparently, that doesn't mean a thing, when using the cmake/ccmake-way with downloaded zip-archive. I always found it saying version "1.0". I found a way to change it, using the date and time of now, in a shell script. But it would make be much more sense, if the official nightly-version-string were to be included.
Interesting.. make -j uses however many jobs you can, whereas -j10 would limit it to 10 jobs. I would have expected that 10 jobs would be high enough. I'll have to try it since taking forever is one of the main reasons why I don't update more often..
Well, I already found if having a problem, but I'm still working on the question, if it really comes from the -j-idea. At my site it doesn't take more than two minutes, using a script that does everything, including mounting my network-drive, where the archive with vanilla- and patch-files lies. But I reverted to -j10 in order to make sure everything works smoothly.

Edit:
I just tried it this way (https://www.ufopaedia.org/index.php?title=Compiling_with_Make_%28OpenXcom%29), which lead my to version »1.0«, no nightly-notification, but the compiling felt like takeing forever...
Title: Re: Upgrading to the nightly
Post by: Arthanor on May 14, 2015, 04:01:58 pm
Apparently, that doesn't mean a thing, when using the cmake/ccmake-way with downloaded zip-archive. I always found it saying version "1.0". I found a way to change it, using the date and time of now, in a shell script. But it would make be much more sense, if the official nightly-version-string were to be included.
If you download the zip-archive instead of using git to get the code, then you cmake/ccmake has no way of getting the git commit. If you use the steps outlined on the wiki with git, then you will have a git repo from which to compile which will give you a hash (the version number looks like 1.0g######).

It also allows you to pull new features into the code even though Warboy/Supsuper did not want to pull them in the main repo, or checkout an older version if you want to run a mod that hasn't updated to the latest nighly yet. But it's a bit more advanced than downloading the zip file.

Quote
Well, I already found if having a problem, but I'm still working on the question, if it really comes from the -j-idea. At my site it doesn't take more than two minutes, using a script that does everything, including mounting my network-drive, where the archive with vanilla- and patch-files lies. But I reverted to -j10 in order to make sure everything works smoothly.

Edit:
I just tried it this way (https://www.ufopaedia.org/index.php?title=Compiling_with_Make_%28OpenXcom%29), which lead my to version »1.0«, no nightly-notification, but the compiling felt like takeing forever...

I first tried the way you just linked, but it did not work for some reason.. While trying to fix it we outlined upon this way with cmake/ccmake. The computer I have at home is a crappy little notebook, so that might be why it's so slow ;) OpenXCom is pretty much the pinnacle of what I can play on this, which is fine by me since it's also the pinnacle of gaming!
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 14, 2015, 04:59:35 pm
I'm beginning to understand a few things. I first tried it like this (https://www.ufopaedia.org/index.php?title=Compiling_with_CMake_%28OpenXcom%29#End-to-end_example), failing. Just now, I tried it by using the line
Code: [Select]
git clone https://github.com/SupSuper/OpenXcom.gitinstead of the one in the link, and it runs just right, giving me the hash. I believe, I will change my skripts, now. =) Just one question: Is there any way, to get a versioning like on the download-page (see screen shot).
Title: Re: Upgrading to the nightly
Post by: myk002 on May 14, 2015, 05:57:17 pm
I'm beginning to understand a few things. I first tried it like this (https://www.ufopaedia.org/index.php?title=Compiling_with_CMake_%28OpenXcom%29#End-to-end_example), failing. Just now, I tried it by using the line
Code: [Select]
git clone https://github.com/SupSuper/OpenXcom.gitinstead of the one in the link, and it runs just right, giving me the hash. I believe, I will change my skripts, now. =) Just one question: Is there any way, to get a versioning like on the download-page (see screen shot).
I added a note to the wiki page about setting up your ssh key so the git clone git@ form will work, and I added a note that if you don't have a github account, you should use the git clone https://github.com/SupSuper/OpenXcom.git form you used.

You can always set the version string to whatever you like by compiling with -DOPENXCOM_VERSION_GIT=whateveryouwant, but I don't think you can set it when using cmake to build.  you can modify the CMakeLists.txt file to make it configurable, but for now it sets it internally according to the git revision and doesn't read anything custom from the config or commandline.  you might be able to get around it by sticking it in your custom CXXFLAGS or something, though.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 14, 2015, 06:26:30 pm
I'm using the steamroller tactics: Automatically changing the
Code: [Select]
#define OPENXCOM_VERSION_SHORT "1.0"line to
Code: [Select]
#define OPENXCOM_VERSION_SHORT "1.0 Nightly <Dateandtime>"by use of a short perl-command in the script. ;D
Title: Re: Upgrading to the nightly
Post by: winterheart on May 14, 2015, 07:47:21 pm
I'm using the steamroller tactics: Automatically changing the
Code: [Select]
#define OPENXCOM_VERSION_SHORT "1.0"line to
Code: [Select]
#define OPENXCOM_VERSION_SHORT "1.0 Nightly <Dateandtime>"by use of a short perl-command in the script. ;D
Pffft.

Code: [Select]
make CXX_DEFINES="-DOPENXCOM_VERSION_GIT=\\\"\ Super\ Awesome\ Edition\\\""

Now you have real CMake powah.

NB: Backslashes required for escaping symbols like space, \ and "
Title: Re: Upgrading to the nightly
Post by: Yankes on May 17, 2015, 05:45:42 pm
myk002 could you add info about your changes to https://ufopaedia.org?
Title: Re: Upgrading to the nightly
Post by: myk002 on May 17, 2015, 07:13:42 pm
in progress.  right now i'm /reading/ all the current openxcom UFOpedia articles before I go making changes.
Title: Re: Upgrading to the nightly
Post by: Tarvis on May 31, 2015, 06:53:23 am
I guess it's a bit late now, but would it be better for OpenXcom to look for UFO's files in the 'XCOM' folder instead? This would make OpenXcom work right out of the box by installing it into the directory of the Steam version, for example, without having to rename it. Of course it's still trivial.

Perhaps it could check either one?
Title: Re: Upgrading to the nightly
Post by: pilot00 on May 31, 2015, 02:50:42 pm
Question: Are the Universal patch files necessary to overright or are they included in the nightly?
Title: Re: Upgrading to the nightly
Post by: 7Saturn on May 31, 2015, 03:29:55 pm
As they patch the vanilla files, they are completely separate, as the vanilla files are never included.
Title: Re: Upgrading to the nightly
Post by: Dioxine on June 06, 2015, 06:45:10 am
How do the soldier names definition work past May 27th Nightly? The format has changed, now it's under Soldiers, cool, but also the formula - it seems that there's no way of replacing stock soldier name files, only adding new ones. Or is there?
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on June 06, 2015, 09:03:10 am
the delete directive works for soldiernames. list it as a filename and it'll delete anything existant in the ruleset listed before but not after it (that would include rulesets loaded prior).
Title: Re: Upgrading to the nightly
Post by: essohbe on June 12, 2015, 06:13:21 pm
Will there ever be a v2.0 or something where people like me who are beyond retarded at tech stuff can just click "install" and not have to worry about doing something wrong with these nightlies?  :-\
Title: Re: Upgrading to the nightly
Post by: DoxaLogos (JG) on June 12, 2015, 06:46:24 pm
Will there ever be a v2.0 or something where people like me who are beyond retarded at tech stuff can just click "install" and not have to worry about doing something wrong with these nightlies?  :-\

As far as I know, 2.0 is when the devs feel confident they have completed support for Terror From the Deep.  So, yes.
Title: Re: Upgrading to the nightly
Post by: Solarius Scorch on June 12, 2015, 06:59:56 pm
Honestly, I think the installing process is a whole level more confusing than upgrading to nightly. There's finding original files, copying the patch, installing updates and so on. With nightly, you just download and copy over some files, so it's beyond me how anyone would have a problem with that.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on June 12, 2015, 07:32:40 pm
Tell that to all those, who run a different OS than Windows... It's not that easy.
Title: Re: Upgrading to the nightly
Post by: Solarius Scorch on June 12, 2015, 09:49:09 pm
Tell that to all those, who run a different OS than Windows... It's not that easy.

Well, it's their choice to do so, with all the pros and cons. But now we're talking about people who have some problems with this stuff, so naturally they want to keep their life easy and not use exotic systems.
Title: Re: Upgrading to the nightly
Post by: 7Saturn on June 12, 2015, 11:54:12 pm
Well, if I had more experience with building packages, I'd create an Ubuntu PPA for it. But last time I read about that stuff, I found it to be a lot of work. So I decided, for me, it's not worth the effort.
Title: Re: Upgrading to the nightly
Post by: hellrazor on June 23, 2015, 09:44:31 am
This is a question that I've given some thought to, and struggled with while designing the new system.  So settle in, this will be a long, technical post.

Within the system as it now stands, you'd handle this by having all resources and a "baseline" mod in one top-level directory, and ruleset changes for the "alternate" version in a second top-level directory.  Users would have either one mod active or both, but never the second without the first.  We will be able to enforce this with requires: tags in the metadata.yml files soon.

The only caveat is that sprite: references in any rules MUST refer to sprites that are defined by either the same rul file or in a rul file in the same directory.  From looking at the AWACS mod, it looks like it won't be a problem (it's ok if both are enabled since AWACS.scenario is a strict superset of AWACS), but in general, a mod can't refer to a sprite offset that is defined by a different mod due to the "index anti-collision" algorithm that is in place.

You'd package your mod like this:
Code: [Select]
AWACS/metadata.yml
AWACS/AWACS.readme.txt
AWACS/Resources/AWACS/*.*
AWACS/Ruleset/AWACS.rul
AWACS.scenario/metadata.yml
AWACS.scenario/Ruleset/AWACS.scenario.rul

and the user would have two directories in the mods dir: AWACS and AWACS.scenario.  They would either enable just AWACS or both AWACS and AWACS.scenario.

The guiding principle for the original design was "make it as simple as possible for the user", and I'd really like to keep it that way.  Of course, the second principle was "make it as simple as possible for mod creators" : )  There were a couple alternate approaches that I considered, but either I or Warboy rejected on the grounds that it was too complicated (but, if they become necessary, we can revisit them):

  • right-clicking on a mod would allow you to choose which of its internal rulesets to load.  this would solve your use case, but introduces a layer of UI complexity that is difficult for a user to "discover".  It also makes it easy for a user to make a mod invalid.  We could get around this by declaring which rulesets are "required" and which are "optional" and which optional ones conflict with other optional ones, but that's all additional complexity.
  • allowing sprite offsets from one mod to be accessible from another mod.  this would allow you to have a "resource-only" base mod with two "business logic" mods that implement your two versions, referring to sprites defined in the first mod.  this would require a good bit of new code and a new resource addressing scheme in the ruleset to indicate that another mod's sprite offsets are being accessed.

My hope is that the existing, simple system is enough, but the conversation for improvement is always open.

Would scenarios also work for different TERRAIN or MAP files or other files or Ressources?
Title: Re: Upgrading to the nightly
Post by: myk002 on June 23, 2015, 08:27:56 pm
Would scenarios also work for different TERRAIN or MAP files or other files or Ressources?
Mods can refer to one another's *files* without any problems.  For example, if some enabled mod (yours or anyone else's) has a file named MAPS/URBAN18.MAP, you can refer to that file in your mod as MAPS/URBAN18.MAP.  This is just as it was before when everything was in the same directory.  The only problem is with referring to another mod's sprite indices.  If you declare the extraSprites in your own mod, even if the images themselves are coming from another mod's files, that will work just fine too, though.
Title: Re: Upgrading to the nightly
Post by: SupSuper on June 30, 2015, 11:12:30 pm
I guess I should mention here that since openxcom_git_master_2015_06_21_1654, mods can have their own language files.
This means you can put all your strings in a "Language/{lang}.yml" file in your mod folder now (like the base game does), which should make translations much easier to manage than the monolithic ExtraStrings method.
Title: Re: Upgrading to the nightly
Post by: Angelus_EV on July 07, 2015, 07:22:59 pm
are there any chance to let the modders post own´s strings to be translated in transifex??
Title: Re: Upgrading to the nightly
Post by: myk002 on July 07, 2015, 11:56:14 pm
are there any chance to let the modders post own´s strings to be translated in transifex??
I don't believe there's any barrier - transifex is free for open source projects.  Just sign up and create an account, then post the strings for your mod.  they won't be included in the main openxcom translation files, but they shouldn't be included there anyway.  you can distribute the translations with your mod yourself just fine.
Title: Re: Upgrading to the nightly
Post by: winterheart on July 08, 2015, 01:24:04 pm
I believe there should be some efforts from openxcom.com mod-site and someone that will be coordinator on transifex. Integration will be little complicating, since modsite threats mods as completed zip-files.
Title: Re: Upgrading to the nightly
Post by: kikimoristan on July 17, 2015, 01:02:13 pm
any ETA on TFTD play?
Title: Re: Upgrading to the nightly
Post by: arrakis69ct on July 19, 2015, 08:14:18 pm
Surprise surprise

Enviado desde mi ECOO E04 3GB mediante Tapatalk

Title: Re: Upgrading to the nightly
Post by: 7Saturn on July 19, 2015, 08:23:12 pm
Gesundheit!
Title: Re: Upgrading to the nightly
Post by: hellrazor on July 25, 2015, 10:06:33 pm
I don't believe there's any barrier - transifex is free for open source projects.  Just sign up and create an account, then post the strings for your mod.  they won't be included in the main openxcom translation files, but they shouldn't be included there anyway.  you can distribute the translations with your mod yourself just fine.

I will give this a try, since i need more language support for my mod. So i just register there and dump my extraString into accordingly named yml files or how?
Title: Re: Upgrading to the nightly
Post by: hellrazor on July 25, 2015, 10:11:27 pm
I believe there should be some efforts from openxcom.com mod-site and someone that will be coordinator on transifex. Integration will be little complicating, since modsite threats mods as completed zip-files.

It would depend, first of all each Mod would need to have proper * .yml files for Language Support instead of useing extraStrings section.

The server could unpackthe zip file, extract the files with strings which need to be translated, put them on transifex. Wait for updates from there and repack them automatically, or mail them towards the modauthor orsomething like this.
Title: Re: Upgrading to the nightly
Post by: Mr. Quiet on July 28, 2015, 03:30:25 am
So, I can bypass 1.0 files, go straight to downloading the nightly, and drag all my XCOM subfolder files into UFO within the nightly? Easy :D, but let me know if I did it right. I'm playing and so far no issues. All those .dll files in the 1.0, that's what confused me, I thought they had to be added into the new nightly folder structure.
Title: Re: Upgrading to the nightly
Post by: NuclearStudent on August 10, 2015, 07:09:57 am
On a vaguely related note, anybody else have a giggle at the development notes?

Specifically stuff like-

Warboy1982, Wed Aug 5 02:17:42 2015
add sanity

even at the cost of my own.

Warboy1982, Tue Aug 4 23:21:11 2015
fml

Warboy1982, Tue Aug 4 22:33:01 2015
objective sites

warboy forgot to set the success flag, what a nooblet.

I feel a little bit bad laughing, but on the other hand, it is hilarious.
Title: Re: Upgrading to the nightly
Post by: Warboy1982 on September 01, 2015, 10:31:50 am
nightly builds page seems slightly broken, i'll keep this link up to date until the issue is resolved.

https://openxcom.org/git_builds/openxcom_git_master_2015_09_02_1039.zip (https://openxcom.org/git_builds/openxcom_git_master_2015_09_02_1039.zip)