HOWTO: Upgrading mods from OXCE 7.15 to OXCE 8.0

[Meridian]

Hello everyone,

we've decided to change the YAML framework
from Yaml-cpp: https://github.com/jbeder/yaml-cpp
to RapidYAML: https://github.com/biojppm/rapidyaml

This should bring significant improvements for you in:
- game startup time (2x faster or better)
- game save/load time (didn't measure yet, but I expect at least 2x faster)
- less silently ignored syntax errors
- less memory (RAM) used

And also other significant improvements for me in:
- game startup time in debug mode (~60x faster) -- no kidding, a big mod starts in 15 seconds instead of 15 minutes!
- no headaches telling everybody which version of yaml-cpp is or isn't compatible with OpenXcom -- we've included the rapidyaml source code directly, so no need to compile or download libs or dlls for every operating system, build mode and star constellation


Just like with everything else, there are also disadvantages.
The mods that worked with OXCE 7.15 may stop working with OXCE 8.0 and will need changes:
1. some cosmetic changes (e.g. removal of trailing whitespace)
2. and some non-cosmetic changes (mostly fixing issues that were silently ignored in OXCE 7.15)

Below are the most common changes that the modders may need to perform.


SPECIAL THANKS to Delian, who has started this initiative, and also implemented most of the changes himself: https://openxcom.org/forum/index.php?topic=12283.0

Cheers,
M.

[Meridian]

1. Convert the "UTF-8 BOM" files to "UTF-8" files

To quote from https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 :
The Unicode Standard permits the BOM in UTF-8, but does not require or recommend its use.

OXCE also implements some workarounds to process the BOM, so strictly speaking you don't have to do this.
But, we do not guarantee everything will work correctly and also provide no support in this area.
We strongly recommend removing BOM from all your YAML files.


How to find files with BOM?
1. first option is to search for the BOM itself (let's say in Total Commander a regular expression \xEF\xBB\xBF)
2. second option is to open a file in a decent text editor (let's say Notepad++) and check the format/encoding (in Notepad++ you can find it in the "Encoding" main menu)

How to convert the files?
a/ open in Notepad++
b/ go to menu Encoding and change from "UTF-8 BOM" to "UTF-8"
c/ save


Example from XcomFiles: https://github.com/SolariusScorch/XComFiles/commit/ff2859dd6188277ada0897c4b1a6c2a4dcbc8bf1

[Meridian]

2. Remove trailing whitespaces

Technically, only some trailing whitespaces need to be removed (for example the ones following ] and } ).
But removing all trailing whitespaces is an industry-standard for decades now.
We strongly recommend removing all of them.


The easiest way is to simply open each YAML file in a decent text editor and let it do the job:
a/ open in Notepad++
b/ go to menu Edit > Blank Operations > Trim Trailing Space
c/ save

If you have many YAML files, you can try automating this with your favourite IDE.

Tip: most, if not all IDEs and text editors offer the option to remove all trailing whitespace automatically on each save. Consider if you want to use such option.


Example from XcomFiles: https://github.com/SolariusScorch/XComFiles/commit/773cfa47c7880b51f2acc53cc2c3a3d6a08316e8

[Meridian]

3. Fix unclosed quotes

Parsing strings with quotes can be tricky, sometimes parsers need help.
RapidYAML cannot (or doesn't want to) parse a string that starts with a quote but doesn't end with a quote (because it thinks it's a typo and there should be a closing quote too).

So instead of:

Code: [Select]

  - 'Akamu

simply use:

Code: [Select]

  - "'Akamu"


Example from XcomFiles: https://github.com/SolariusScorch/XComFiles/commit/b31a1d3aee2a8ea521d604592d7342753e18e823

[Meridian]

4. Fix comments without preceding whitespace

A comment in YAML must start with a space and # (unless the whole line contains nothing else than a comment; then it can start simply with a # without a space).
Simply add spacing where required.

So instead of:

Code: [Select]

              - position: [9, 18, 0]#my comment

simply use:

Code: [Select]

              - position: [9, 18, 0] #my comment


Example from XcomFiles: https://github.com/SolariusScorch/XComFiles/commit/37cc3107cd5ed9b177c9c46c672a76f6efeb9e8e

[Meridian]

5. Fix syntax errors

Most typical ones are:
- a number expected, but string found
- an integer number expected, but a decimal number found
- a number expected, but a list found
- a colon missing
- a tab used (YAML doesn't allow any tabs, not even in comments)
- etc.

Some examples:
https://github.com/SolariusScorch/XComFiles/commit/af25a02e408934edb55e172db7fa3e80cbf21bfb
https://github.com/SolariusScorch/XComFiles/commit/6060853d56cced621ca2d564587a02dd7c7d0ae1
https://github.com/BeatAroundTheBuscher/ROSIGMA/commit/f3d669e908b94c24831ee59cec1a865ff5d79ed2
https://github.com/BeatAroundTheBuscher/ROSIGMA/commit/15d973a484f59d3b580576bbf8d79a66f28c3683