VSCode OpenXcom Modding Tools

[pedroterzero]

Sounds useful. Many thanks for your work!

Image

Cheers for the support guys I hope you try it out sometime, or if you already have, you find it useful!

In other news, there's now a discord channel courtesy of efrenespartano: https://discord.gg/2WjVTvJcbQ

[scarf]

this is by the far one of the most useful modding tools I've ever had. great work!!


p.s - that font is sleek. what's it called?

[pedroterzero]

this is by the far one of the most useful modding tools I've ever had. great work!!

Thank you for the kind words, it is appreciated! If you have any suggestions about how I could improve it even more, let me know!

p.s - that font is sleek. what's it called?

If you mean the font in VSCode the demo videos, I use Courier New. It's a pretty old default Microsoft font.

[scarf]

Thank you for the kind words, it is appreciated! If you have any suggestions about how I could improve it even more, let me know!

I guess the ability to ignore some of the warnings would be great. Sometimes the ruleset works as intended but linker sees that as error. maybe adding #ignoreLinker  to end of line to ignore certain properties for checking?


+ also highlight syntax for loop
for example word 'loop' and 'var' not highlighted on

Code: [Select]

loop var i 10;
    debug_log i;
end;

[pedroterzero]

I guess the ability to ignore some of the warnings would be great. Sometimes the ruleset works as intended but linker sees that as error. maybe adding #ignoreLinker  to end of line to ignore certain properties for checking?

I'd prefer to fix them, as in that case the tool is reporting false positives. Can I check out your rulesets somewhere so I can see what's going wrong?

+ also highlight syntax for loop
for example word 'loop' and 'var' not highlighted on

Code: [Select]

loop var i 10;
    debug_log i;
end;

I'll look into updating the syntax highlighting! I based the highlighting on a version of OXCE where loops weren't a thing yet

[scarf]

I'd prefer to fix them, as in that case the tool is reporting false positives. Can I check out your rulesets somewhere so I can see what's going wrong?

these all are the false positives despite the code working as intended; linker will tell on attachment...
'cannons-conventional.rul' (STR_CANNON is pre-defined, not sure about STR_NONE)

Code: [Select]

"STR_NONE" does not exist (craftWeapons.clip) for STR_CANNON_UC
'STR_CANNON_UC' launcher 'STR_CANNON': item does not exist. This will cause a crash on loading OpenXcom!'terror-unit-weapons.rul' (bulletSprite 10 should be vanilla blaster launcher's bulletsprite)

Code: [Select]

"10" does not exist (items.bulletSprite) for STR_SECTOPOD_AUX_WEAPON
Hint: this needs a sprite in Projectiles with spriteID 350 (10 * 35)
        (see documentation by hovering over "bulletSprite:")
'coverall.rul'

Code: [Select]

"MAN_0" does not exist (armors.spriteInv) for STR_COVERALL_UC


'awacs.rul' (on extrasprites INTICON.PCK is made of six sprites starting from 26;)

Code: [Select]

"26" (also "27", "28") does not exist (crafts.sprite) for STR_HAWKEYE (also STR_AWACS, STR_DARKSTAR)
Hint: this needs THREE extraSprites (see documentation by hovering over "sprite:"):
        - One in INTICON.PCK with spriteID 26
        - One in INTICON.PCK with spriteID 37 (26 + 11)
        - And one in BASEBITS.PCK with spriteID 59 (26 + 33)
'mapScripts_CMPV.rul' (I'm not sure about this one, but I've tested the maps that was supposed to get segfault and they were okay, looks like groups can be arbitary)

Code: [Select]

'Group '4' does not exist in terrain for RICE_FARM_SCRIPT. This will cause a segmentation fault when loading the map!

+ ADDED


Units-Alien.zip contains alien deployment data, which was split into:

• data (where alienRank is stored)
• width, length, height
• civilians, alert (and background), briefing, markername, duration and despawnpenalty

since the only part where splitting is not allowed is data, files above works fine and without bug (afaik), however they're spewing out like, 40 fatal errors.

[pedroterzero]

stuff...

Thanks for reporting all of these, I have fixed them and will push them to marketplace soon as 0.8.2. There was only one issue (below) that was not actually a false positive but the reported problem did not match with the reality.

'mapScripts_CMPV.rul' (I'm not sure about this one, but I've tested the maps that was supposed to get segfault and they were okay, looks like groups can be arbitary)

Code: [Select]

'Group '4' does not exist in terrain for RICE_FARM_SCRIPT. This will cause a segmentation fault when loading the map!

This one does not actually cause a segfault. It does happen with addLine, but for addBlock it just gets ignored. I have updated the error message in this case to reflect that. For addLine it will still report a segfault.

[spawi]

This is an awesome extension, thank you!

I have a suggestion re performance.

I loaded X-COM Files with this extension and it took, I think, over 15 minutes to load all rulesets, and my laptop fan was going bonkers during this entire time. X-COM Files has some massive rulesets, e.g. research_XCOMFILES.rul is over 24000 lines long, and the loading progress was stuck on it for the majority of the time it took to load.

Is there a way to speed up this loading process? E.g. cache it, and later diff and update incrementally? If I reopen VS Code I believe it will try to load everything anew.

[pedroterzero]

Is there a way to speed up this loading process? E.g. cache it, and later diff and update incrementally? If I reopen VS Code I believe it will try to load everything anew.

Thanks for the kind words! There is already cache on a file basis, once a .rul file has been parsed, it should not have to be parsed again unless it changes (or the extension gets an update). However one change means that entire file needs to be parsed again, due to my current implementation.

I need to take a look at why it's taking exponentially long to deal with larger files like in XCF, I am not sure. But I would probably need to do a pretty big overhaul on the main parsing bit of the code. It's one of the first things I wrote for this extension, and knowing what I know now I would have done it differently. It's not the easiest bit for me to work in though.

I'll try to have a look when I can if there's any quick band-aid fixes I can apply, I'll let you know here if I do.

[spawi]

Thanks for the kind words! There is already cache on a file basis, once a .rul file has been parsed, it should not have to be parsed again unless it changes (or the extension gets an update). However one change means that entire file needs to be parsed again, due to my current implementation.

I need to take a look at why it's taking exponentially long to deal with larger files like in XCF, I am not sure. But I would probably need to do a pretty big overhaul on the main parsing bit of the code. It's one of the first things I wrote for this extension, and knowing what I know now I would have done it differently. It's not the easiest bit for me to work in though.

I'll try to have a look when I can if there's any quick band-aid fixes I can apply, I'll let you know here if I do.

Indeed, upon next open the rules were loaded in about 10 seconds. Awesome! I underestimated your implementation, sorry!

I totally see how this can be a highly nontrivial issue. As the saying goes, there are three hard problems in software engineering: cache invalidation and off by one errors

Nevertheless, I think that with fast reload without edits this works quite well already. For example, if I need to make edits to one of these humongous files, I could batch them and don't reopen VS Code until I am done, then reload. Also, it is an argument for splitting these files into smaller ones (I do hope OXCE supports reading rules of one type from multiple files).

I think your tool adoption would benefit from being more explicit about this caching. I think some people might give up before the 15 minutes for initial load are up. If the tool would somehow report how long it will take, and clarify it will be fast without edits, and reload only specific files upon edit, that would motivate people to give it a proper try.

[pedroterzero]

Indeed, upon next open the rules were loaded in about 10 seconds. Awesome! I underestimated your implementation, sorry!

I totally see how this can be a highly nontrivial issue. As the saying goes, there are three hard problems in software engineering: cache invalidation and off by one errors

15 minutes, and then 10 seconds cached is still too long, obviously I still hope to be able to do something about it sometime.

Nevertheless, I think that with fast reload without edits this works quite well already. For example, if I need to make edits to one of these humongous files, I could batch them and don't reopen VS Code until I am done, then reload.

Good to hear. You could try to see how long saving a single file works. The moment you save is the moment it parses the file again. Differential parsing could help in this matter, but I'm not sure how to approach that (yet). The yaml library I use offers no such functionalities, I'd have to find something generic, or set it up myself.

Also, it is an argument for splitting these files into smaller ones (I do hope OXCE supports reading rules of one type from multiple files).

Yes, that is very possible. You can basically split things anyway you like (as far as I'm aware), and that would certainly help the problem. It's a workaround and it obviously doesn't directly address the core problem, but it will help.

I think your tool adoption would benefit from being more explicit about this caching. I think some people might give up before the 15 minutes for initial load are up. If the tool would somehow report how long it will take, and clarify it will be fast without edits, and reload only specific files upon edit, that would motivate people to give it a proper try.

I think the larger problem is that not too many people are aware of it, and secondly that people are quite content just using notepad++ for their modding. That said, I could drop it in the README somewhere. For most mods that I've tried it with, it's not really a problem though. It's the really large ones (there's only XCF, FMP and X-Piratez that are like that I think?) that are causing the long load times. There is something in my parser that does not scale, clearly 

[pedroterzero]

I just published 0.8.2 to vscode marketplace. Not a very exciting release with many new features, but a great number of fixes/tweaks.

On the roadmap currently:

• Ruleset spreadsheet editor (allows two-way editing of .rul files as spreadsheets either in vscode itself or using Excel)
• Y-script syntax validation
• Missing translation detection
• Missing files detection (sprites/sounds)
• other stuff/li]
  

This is the changelog for 0.8.2:

• Added missing vanilla armor sprites
• Fix sprite index checking with subX/subY for more sprite types (INTICON)
• Do not retrieve keys from aliases (they only need checking once), fixes problems
• Store logic data for vanilla assets (but don't check them), prevents false positives when checking them
• Improved message in case of mapblock that does not cause segfault but will just cause block to be ignored
• Fix autocomplete for line that is also start of an array entry
• Improve duplicate error message (use related information, #35)
  
• Fix relatedlogic fields overwriting existing typeLinks
• Added missing type link for alienMissions.waves[].ufo
• Added missing lt & le in y-script highlighting
• manufacture.requiredItems add crafts as valid type
• Added missing stringType (alienDeployments.objectiveFailed)
• Combine alienMissions data from multiple files
• Ufopaedia image no longer required for type 3, apparently
• Parse comments in extraSprites/extraSounds (so duplicates can be ignored)
• Enable subX/subY for BIGOBS and FLOOROBS
• Add hitMissSound vanilla ids
• No longer check ufopaedia.weapon (it's a translatable string)
• Properly handle refNodes in logic checkers may need to add in other positions)
• Reload Language/*.yml on changes, so translations get picked up #26
  

[Hobbes]

Just to say that I've tried pedroterzero's tool recently and I'm completely sold to it just for parsing the rulesets and fixing bugs I had no idea they were there.

For large mods, this is a must have if just to periodically check your rulesets and increase mod stability. And it will save you a lot of pain and time lost from bugs not to easy to find.

[pedroterzero]

Just to say that I've tried pedroterzero's tool recently and I'm completely sold to it just for parsing the rulesets and fixing bugs I had no idea they were there.

For large mods, this is a must have if just to periodically check your rulesets and increase mod stability. And it will save you a lot of pain and time lost from bugs not to easy to find.

Thank you! That is very high praise indeed coming from one of the most experienced modders out there. I appreciate it very much.

[pedroterzero]

I have just released the v1.0.0 (first "stable") version of my extension has been released on the marketplace. It includes the new CSV/spreadsheet editor and many fixes/corrections. Many thanks to all the testers/users of this exntesion that helped me get it over the line. Visual studio code should auto update you to this latest version.

Here is a demo of the new spreadsheet editor: