XPirateZ Squad Analyzer (Help Wanted)

[ivandogovich]

The learning curve from OXC to XPirateZ is steep!  Trying to figure out how to get the best out of your gals, (especially in the early game) added to the huge variety of weapon types, can make your brain burn.  I tried setting up some spreadsheets to help me calculate which weapons benefited which gals the most.  I found this most useful for melee weapons because they stats they use can be so varied (bravery, reactions, melee, etc).

Then I got in touch with Imeryak to ask him for help pulling the data from my gals into an easy spreadsheet.  I could use this to import the stats into my analyzer.
Tool here: https://drive.google.com/open?id=0B3LA7DXD41YYQzdYVmRMRjRhN2M

App grabs all soldiers from all bases, their names, missions, kills and current stats.
There are two buttons - one to open save files and one to save CSV files with soldiers stats. Those can be opened with almost any spreadsheet editor, including google docs.
To install - run setup.exe from publish folder or try sde\bin\Release\sde.exe to run it without installation.
Source code is included, so you may look at it and compile it yourself with Visual Studio 2013 Community Edition.
I am sure there will be bugs, so contact me when you find one.

Now on to the Squad analyzer.  We need to be able to define the weapon attributes (power, damage type, TUs, modifiers, etc).
Then we should have a sheet where the soldier's stats are matched against them allowing for analysis.  Such things as average expected damage (before armor), and maybe even maximum damage per turn?? (i.e. are 6 swings with a machete better than 3 with a cutlass?, etc).

I have a start of a google sheet that does some of this.  It has early game melee weapons, and sheets for the squaddies.  The Sheets call from the weapon sheet and compare them.  https://drive.google.com/open?id=1HvYn4zLRgPLozx0XUwmdb1WKPKxWFKINLP74wYks78U
Overall, I'm not convinced this is best format, but I'm putting it out here as a starting point. 

Spreadsheet gurus, data nerds, and creative thinkers are welcomed with open arms to this project!

[Arthanor]

Ivan!! You cost me my evening!! Here's what I have:

https://docs.google.com/spreadsheets/d/1kyo9x_RGeVEer73qqgUxvgsqBcScuQRAcA5j09bhOqw/edit?usp=sharing

In theory, you could have a program/script which can write a csv for all weapons by parsing the ruleset, and all gals by parsing the save.

The only pain is implementing more weapons because of the way the spreadsheet tries to increment all references instead of just the proper ones, but.. you can write a script that does that too! (ie populate your csv).

So basically, with 3 scripts you should be able to:
1 - Get all weapons
2 - Get all gals
3 - Get a spreadsheet with how every gal does with every weapon

You can also write a script to:
4 - Get all units
5 - Get all armors
6 - Get a spreadsheet with how tough every unit in the game is
7 - Combine the gals & weapons spreadsheet with the units toughness one into THE ULTIMATE PIRATEZ SPREADSHEET!!! ... OF DOOOOM!! that allows you to see how all your gals would do with all weapons against all units. Obviously, that might be more data than an average brain can process... indeed, just seeing how gals do with all weapons is already quite a bit of data. This is why spreadsheets are not great analysis tools. Even Piratez breaks their ability to be really meaningful.

Navigating through the full list to see just the best weapon for a gal (or even worse, just the best weapon for a gal for a specific enemy) will be challenging. So..:
8 - write a parser for the ultimate piratez spreadsheet of doom that allows you to select a gal, a role (scout/grenadier/breacher/etc.) and a target, to spit out the best melee and ranged weapon.

Then you barely need to play the game.. and if Meridian integrates is in OXCE+, we'll get OXCE++ : the game that almost plays itself...

[onionradish]

You're right about the challenge in evaluating equipment options!

It's tough when you're starting out (like me) because you don't yet have experience with the new gear in use. I can see it getting even tougher as the game progresses and there are even more options to consider from captured gear and the black market -- plus a growing crew of gals.

I started some number-crunching earlier this week to help me decide starting gear. I'm using a rough, exploratory script in Python to read directly from the X-Piratez files and my save games. Reading from the files gives access to all the weapon damage stats and rules -- which can be pretty complex -- access to all the gal stats, and all equipment in bases, ships and equipped or carried by each gal.

Here's one example of rough results from that script using the "Ivana" gal stats from ivandogovich's spreadsheet for comparison:

Code: [Select]

--------------------
Saved game version: Extended 2.9+
Available weapons sorted by highest total damage
Weapon name: base_dmg + bonus_dmg + alter_dmg = total_dmg
Weights in square brackets

IVANA  [34]  (left: Smoke Grenade [-], right: Cutlass [7])
--------------------
bravery: 10, firing: 53, melee: 87, reactions: 65, strength: 34, throwing: 36
--------------------
Hammer: 60 + 43.1 + 0.0 = 103.1 [24]
Ax: 50 + 32.5 + 0 = 82.5 [12]
Cannonball: 80 + 0 + 0 = 80 [3]
Cutlass: 35 + 31.0 + 0.0 = 66.0 [7]
Ball Bat: 25 + 10.2 + 26.4 = 61.6 [7]
Launcher Grenade/HE: 60 + 0 + 0 = 60 [3]
Black Powder Bomb: 60 + 0 + 0 = 60 [4]
Rapier: 20 + 34.8 + 0 = 54.8 [6]
Short Sword: 25 + 29.5 + 0.0 = 54.5 [5]
Billhook: 40 + 13.2 + 0.0 = 53.2 [8]
Fistycuffs: 0 + 37.8 + 13.23 = 51.03 [3]
Harpoon Stun Bolts: 50 + 0 + 0 = 50 [3]
Flintlock Gas-Balls: 30 + 0 + 18.0 = 48.0 [2]
Boarding Gun Ammo: 46 + 0 + 0 = 46 [8]
Machete: 20 + 24.2 + 0.0 = 44.2 [3]
Dagger: 20 + 24.2 + 0 = 44.2 [3]
Handcannon Rounds: 44 + 0 + 0 = 44 [2]
Pipe: 15 + 20.4 + 0.0 = 35.4 [5]
Chainsaw: 35 + 0 + 0 = 35 [9]
Fuso Knives: 0 + 32.4 + 0.0 = 32.4 [4]
Assault Rifle Clip: 30 + 0 + 0 = 30 [3]
Flintlock Balls: 29 + 0 + 0 = 29 [2]
Banana Clip: 28 + 0 + 0 = 28 [3]
Blunderbuss Shells: 24 + 0 + 0 = 24 [3]
Handle: 10 + 10.8 + 3.12 = 23.92 [4]
Rope: 5 + 12.2 + 3.44 = 20.64 [4]
.8g Buckshot: 20 + 0 + 0 = 20 [2]
Molotov Cocktail: 20 + 0 + 0 = 20 [3]
Hunting Bow: 0 + 18.0 + 0.0 = 18.0 [6]
--------------------

Or a comparison of who is best at using a particular weapon (example melee and thrown):

Code: [Select]

Cutlass
Jazz: 74.2, Cable: 70.8, Jinx: 68.4, Shadow: 67.6, IVANA: 66.0, Bonzai: 65.8, Freefall: 63.8, Stalker: 61.0, Wardog: 60.6, Specter: 59.8, Pixie: 59.8, Zealot: 58.2, Sideswipe: 58.0, Saint: 57.6, Nova: 54.0, Renegade: 51.2, Luna: 48.2

Fuso Knives
Stalker: 53.1, Saint: 52.2, Renegade: 50.4, Wardog: 50.4, Cable: 49.5, Jazz: 46.8, Freefall: 41.4, Sideswipe: 41.4, Zealot: 41.4, Nova: 39.6, Jinx: 39.6, Pixie: 35.1, Shadow: 33.3, IVANA: 32.4, Luna: 29.7, Bonzai: 29.7, Specter: 28.8


From working with the stats, and poking through source code, here's what I see so far:

• Given how frequently X-Piratez is evolving, the number of items, gals, stats and how complex the calculations can be, I think we'll need a tool or script to do the work so it can be run against the latest game data and save files, with results from that tool dumped to a spreadsheet, HTML, TXT or whatever makes sense.
• Limiting the analysis to just available (or otherwise researched/unlocked) equipment is helpful because it prevents spoilers and doesn't clutter the output results with stuff you don't have or can't use. That's easily done, as is limiting analysis to just what's equipped or being carried.
• Accuracy may need to be accounted for -- which is another level of complexity due to the different types of shot. Without it, even though there is variation in the ranking of weapon damage for each gal, the impact of each gal's skills may affect the determination of her "best" weapon.
• Weapons with ammo have some extra complexity because the stats from both gun and ammo need to be considered together. It looks like ammo has the damage stats, and the gun has accuracy stats; I'm not sure if they interact more than that; right now I'm only evaluating the ammo.
• Some sort of DPS-like ratios that take damage, accuracy, TU/energy cost into account seems plausible, but some quick calculations by hand seem "off," like snap shots doing nearly double the "potential damage ratio" as an aimed shot using a simple DMG * ACC / TU forumla. I'd be interested to see any calculations or formulas that could be tried or make more sense.
• The current damage calculation I'm doing only accounts for direct damage to "health" (or tries to) but there are other stats that can be damaged (stun, armor, etc.), and that probably needs to be considered as well. I will probably rework my calculation to return damage for each stat instead of just health.

I've tried to parse the logic for the accuracy and damage calculations from the source code, but haven't got it completely figured out. There are some weapon stats that are sort of obscure, and I'm not completely sure I understand how all of them interact with a gal's skill stats. I'd welcome any details, info, or discussion on how accuracy and damage (of all types) is calculated -- especially for some of the unusual weapons. 

EDIT: In many ways, I'd even prefer result output that's more a general recommendation for a gal, essentially taking the statstrings just one level further, not even showing the numbers: "Ivana is probably best with a melee weapon and would do well with MELEE_A or MELEE_B. As a secondary or alternative, consider GUN_A, GUN_B or THROWING_A." Some of the fun is in making the choices that suit your playstyle and experimenting with how it works in combat; for me, the main goal would just be to narrow the range of choices from dozens to a few.

[ivandogovich]

I started some number-crunching earlier this week to help me decide starting gear. I'm using a rough, exploratory script in Python to read directly from the X-Piratez files and my save games. Reading from the files gives access to all the weapon damage stats and rules -- which can be pretty complex -- access to all the gal stats, and all equipment in bases, ships and equipped or carried by each gal.

<snip>


I've tried to parse the logic for the accuracy and damage calculations from the source code, but haven't got it completely figured out. There are some weapon stats that are sort of obscure, and I'm not completely sure I understand how all of them interact with a gal's skill stats. I'd welcome any details, info, or discussion on how accuracy and damage (of all types) is calculated -- especially for some of the unusual weapons. 

EDIT: In many ways, I'd even prefer result output that's more a general recommendation for a gal, essentially taking the statstrings just one level further, not even showing the numbers: "Ivana is probably best with a melee weapon and would do well with MELEE_A or MELEE_B. As a secondary or alternative, consider GUN_A, GUN_B or THROWING_A." Some of the fun is in making the choices that suit your playstyle and experimenting with how it works in combat; for me, the main goal would just be to narrow the range of choices from dozens to a few.

Great comments, analysis, and feedback!  Any help developing a tool to harvest the data from rulesets would be great!  I really like your idea for the "Recommended Weapons" as a nice final, simple product without all the number crunching.  I'm not really sure how to get there from here.  (Aside from building a complete utility program as a stand alone (X-Piratez Squad Util) and that is way beyond any programming ability I would have).  If we came up with a way to do so, it would still be good to have the more in depth information available for the number crunchers.

Let me get back to discussing some of your specific questions.

I've tried to parse the logic for the accuracy and damage calculations from the source code, but haven't got it completely figured out.

Some of the formula's are straight forward as you have noticed.  (Accuracy + (Melee*.5)), etc.  Others are exploiting a format that Yankes developed to add a ton more flexibility.  I'll see if I can parse it correctly, but at least you could follow up with Yankes if I can't explain it right.

From Yankes' OXCE Readme:

Code: [Select]

damageBonus: #used by ammo.
      #each bonus can be writen as: `s: x`, `s: [x]`, `s: [x, y]` or `s: [x, y, z]`.
      #where each value represents coefficient of polynomial `x * s + y * s^2 + z * s^3`.
As an example the Fuso knives use this as part of the accuracy modifier:

    accuracyMultiplier:
      firing: 0.0
      throwing: [0.0, 0.025]

I parsed this as :
(0 * Throwing) + (.025*(Throwing^2)

Overall, I think these open up the ability to change the curve of effectiveness as stats improve, but I'm not exactly sure.

Now, onto your question that you sent via message:  (and thanks, BTW!!)

Any thoughts on the kinds of info you'd want to see?

I think the average weapon analysis should include some of the following.
1. Total damage should indicate the Average Damage per swing.  (With 0-200% as the range, I've just left it as 100%).
2. There should be some consideration for likelihood of hitting.  20% chance to hit, vs 80% chance to hit is a big  difference.
3. Possible Damage Per Turn: (if you can expend all of your TUs for attacks, multiply your # of attacks (TUs/AttackCost) times the Average Damage per Attack.  This may make faster weapons more viable than slower, heavier hitting, ones (i.e. Machete vs Hammer on an unarmored target)
4. Should the Possible Damage Per Turn be multiplied by the likelihood? (i.e. * 80% hitchance, etc)?

Somethings along these lines would be nice for the number crunchers.

Am I making any sense? (I'm not sure) XD

[yrizoud]

IMO, the important part is that in the end, each gal has a "favorite" weapon that you can equip her with, and it's not a complete misuse of her skills.
Weapon speed must be taken into account. Since you're comparing 2 attacks of one weapon versus 4 attacks of another (for example), target armor must be taken into account : it's substracted 2 times in the first case, and 4 times in the second case. An arbitrary armor value must be chosen. Or if you like overkill, do one calculation for a mostly harmless opponent, one for a moderately armored, one for a heavy, and average the three.

For effective damage, I generally multiply all these things together :
1) %chance of hitting
2) %chance of doing ANY damage on hitting
3) average damage of the attacks which overcome armor
4) a factor depending on the practical number of attacks in a round

For melee attacks speed, 4 could be counted by assuming the girl spends 1/2 of her TUs getting to the target, and the rest attacking. Count the attacks she can make, keeping the decimals.

[ivandogovich]

  

Yeah, I've pretty much tried to avoid the Armor side of things.  There are so many variables involved there, that we would really need to get into a "avg damage vs. Target A (i.e. Trader G.O.), avg damage vs. Target Q (i.e. Cyclops), avg damage vs. Target ZZ (i.e. Cyberdisc).  Because each of the resistances are so varied, it becomes mind boggling.

I think that by just focusing on what amount of damage they can put out in a single attack, you can get a sense of if "28 cutting" damage might hurt that unarmored vs. light armored vs. heavy armored dude.

I'm not saying that approach is wrong, I'm just not sure that it might blow the scope of this out of the water.

[Arthanor]

No comments on my spreadsheet? 

It already shows TUs, TUs/Attack (to get # of attacks/turn), chance to hit and damage applied given all the stat modifiers.

The avg/turn is # of attacks * % to hit * average damage
The max/turn is # of attacks * max damage (= you're hitting with every attack and rolling 200% damage for hit)

I think with these two you can a pretty good idea of what the weapons will do.

In this case, using Ivana as an example, you see that, of the weapons I entered, a rapier is the most reliable choice (because melee is good and strength not great), with cutlass a close second that could do more damage if you are lucky and get few misses (as shown by the cutlass' max damage being higher). Shooting wise, due to poor shooting accuracy, it's better to try to set her up for many rifle snap shots than anything else (and the sniper rifle is obviously a very poor choice).

As it is, the spreadsheet doesn't take armor into account (or armor piercing) but it would be doable. I picture it as a one value thing to set: You put a number in the armor cell and that is substracted from all damage. negative damage tells you that it won't do anything on average. Negative max damage that it won't ever do anything. The user can change the armor value depending on what they are interested in and see the effect.

[ivandogovich]

No comments on my spreadsheet? 

Heh! I was just looking at it!  Looks great!  (Better than my sorry attempt!)

[Arthanor]

I think it should be possible to write something to properly populate it with weapons and formulas (I hate that dragging down formulas increments everything, instead of being able to determine which cell references should be.. ie no incrementing the cell references for the gal's stats)

Reading a ruleset in a proper table format shouldn't be too hard, but I've not programmed yet in a language that would be well suited for the task (and free sharing) :/ A python tool like Falko's would probably be the best.

[ivandogovich]

At a minimum, I'll probably tackle fleshing out the Weapons Table by hand at least for Melee weapons.  I think your sheets are strongest, so thats a great format.

I've some other Bootypedia projects first (Auxilliaries and Armor updates) so if someone can come up with a ruleset parser to grab weapon data before I get to it, that would be awesome.

[yrizoud]

(I hate that dragging down formulas increments everything, instead of being able to determine which cell references should be.. ie no incrementing the cell references for the gal's stats)

Sorry if I misunderstand or if you already know it, but Google Sheets lets you decide if your references are relative (the usual), or absolute (copying or dragging formula keeps the same row/column). It's the same syntax as in Excel, and the key F4 is a shortcut to toggle between all 4 possibilities :
=A1     <== relative in X and Y
=$A$1   <== absolute in X and Y
=$A1    <== relative in X, absolute in Y
=A$1    <== absolute in X, relative in Y

[Arthanor]

 

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

... (reconstitutes blown mind)

!!!!!!!!!!!! 

Nope, I didn't know! I am not a spreadsheet user (besides pet projects like this), my work uses Matlab (which is not appropriate for this kind of stuff).

This is gold man.. GOLD!

I'm so happy I complained about this Thanks!

[onionradish]

Great to see some discussion and ideas!

I've done some continued work on my script while playing and have finally figured out the source code damage calculations for weapons.

Currently, what I'm doing is stand-alone and uses Python to parse and process a save game file for gal/base/ship info against the core X-PirateZ data. I don't know whether or not similar capabilities could be integrated into existing capabilities of XcomUtil to show results in game or not. My thinking is that first we figure out the kinds of analysis we want, and then how to perform and present it. It would not be difficult to spit out a nice interactive HTML report for example, so anyone with design/aesthetic ideas should also chime in. CSV or other formats are also easy.

For each gal equipped with each available++ weapon, I can now calculate the average/baseline* potential damage directed towards an enemy's health, armor, stun, and other stats.

    * Potential damage ranges depend on the weapon's rules and/or the user's choice of UFO/TFTD damage method at the options level and are actually calculated in battle by RNG as a percentage of the baseline (0-2x, 0.5x-1.5x, etc.). Currently, I'm only calculating the baseline/average damage. I may later implement ranges so weapons with higher damage variability are distinct from those with more consistent damage.

    ++ Available weapons are considered a weapon that is is "seen" in bootypedia, is "in stock" at a base/craft, can be bought from black market, or manufactured. It's possible to sub-divide what's "available" as part of recommendation (for example, weapons you can't buy on the black market and have to manufacture could score differently).

My current script is not calculating accuracy or range, but I am investigating how that might work for recommendations. Accuracy/range and similar stats would obviously factor in to a "recommendation" on weapon. As an example, using an early game/gal save, the "Officer's Pistol" offers slightly less damage than the Flintlock, but the "OP" has higher accuracy and higher number of rounds loaded for similar TU and weight, etc. regardless of the gal's firing skill, which may make it a better overall choice.

I'm ignoring all real-time-battle-dependent effects (crouching, injury, morale, etc.). I'm not considering potential enemy type, armor or resistance. My thought for now is that those considerations are up to the player to decide or deal with as they occur. The "potential" seems like all that's possible to optimize, given that a particular target may or may not be armored, may or may not be resistant to a particular type of damage, etc.

From using the info while playing, I'm seeing that the need to evaluate the "best" weapon isn't all that frequent. Mostly it's helpful for figuring out what new recruits are good with and periodically checking after battles, research or other milestone to see whether something has changed.

The more ideas or formulas on how to calculate best/recommended weapon, the better -- please send them because they're easy to add to the script for discussion of the results!

[R1dO]

Nope, I didn't know! I am not a spreadsheet user (besides pet projects like this), my work uses Matlab (which is not appropriate for this kind of stuff).

A multitude of different stat depended attack values vs a multitude of armor values and target health.
If that is not a recipe for a nice surface plot, what is?

[Arthanor]

Oh, sure, a nice plot of kill likelihood as a function of weapons and targets for a given gal?

But I meant in term of sharing the actual program and making something approachable

Looks like onionradish is getting somewhere though!