[DONE][Suggestion] Compact save file formatting

[Meridian]

How long does it take for you to start oxce+xpz normally, and how long with debugging enabled? What factor are you seeing in game performance decrease between the two?

Normally, XPZ starts in 9 or 10 seconds.

In debug mode, I wanted to time it for you precisely, but it crashed on me after 11 minutes and 47 seconds (without starting)... so, at least that much. Usually, I just go and do something else for 15-20 minutes and return when it's started. I go to sleep now, I can measure it again tomorrow.

There is only one factor: the debug mode itself. I use Visual Studio, maybe debuggers in different development environments can be a little faster... but not by much. It will still be the same order of magnitude in all others.

PS: for illustration I measured at least the vanilla xcom precisely: normal startup = 2 or 3 seconds; with debug mode = 30 seconds

[Delian]

After installing a few things... my timings are:
0:07 to start XPZ normally
1:17 to compile in VS2019
2:41 to run debug build game (XPZ)
0:15 to load game (1.5MB sav in battlescape)
It's not some state-of-the-art machine. Just a 2yr old Win10 / 10700 / 16GB DDR4 / 512GB SSD. Under 500€ in total today.
I've read that there are some ways to improve C++ debug build performance in VS, but I'm sure you already know more about that than me. /ZI is already /Zi. /O1 didn't make a difference. /MD fails to compile. /RTC1 didn't make a difference. So, the only thing I can suggest is to upgrade your machine.

Considering how slow debugging is for you, you should be trying to avoid using it to debug common problems people have. In the example you provided, one of the points I was trying to make is that, the problem could've been diagnosed without ever running a debug build.

[Meridian]

I have a good computer, with slightly better (edit: worse?) specs than what you wrote above.

I have never seen a compile of OXCE in 1:17 on any of my computers, on any of my operating systems, on any dev environments... and I tried many, as you can see in the compilation and cross-compilation guides.
Also just look at the github compile times, or compile times of other build farms that several other people have set up. None of them compiled it in such time.

I don't want to guess what you did "wrong"... maybe it was not a full compile, maybe it was incorrectly measured, I don't know...

edit: probably difference in cpu cores involved in compilation

Let's ask others what their times are... I don't have the strength or motivation to argue here...

[ohartenstein23]

Haven't done it in a while, but I kept a RelWithDebInfo build for running in debug mode so I didn't have to compile or if I did it was not a full rebuild it'd be closer to Delian's times. I was running it through gdb though instead of VSCode, so it was probably more difficult to track parameters on at runtime.

[Delian]

Ok, here's a video. Am I doing something wrong? Is Clean Solution and then Build Solution not a full rebuild?

My best guess is that it builds so fast because i7-10700 has 16 core threads, and cpu usage is 100% during building, so 16x the speed. However, that doesn't explain why running the debug build of the game takes so long - that's a single-thread operation.

[Meridian]

yeah, probably the cores
I have only 4-core i7 with 32gb ddr4 and 1tb ssd, win10, vs2022

[Ethereal]

I really like the recent change of making .sav files more compact. So I'd like to suggest a few more ideas to achieve better formatting in the .sav files.

I don't understand shit about programming, but I've been talking about a significant decrease in the speed of saves \ loads caused by file overflow with unnecessary information (mainly due to statistics of soldiers) for a long time. And I am very glad that at least some progress is planned in this direction and work has begun. I really hope that you will be able to solve this problem.

[Yankes]

I don't understand shit about programming, but I've been talking about a significant decrease in the speed of saves \ loads caused by file overflow with unnecessary information (mainly due to statistics of soldiers) for a long time. And I am very glad that at least some progress is planned in this direction and work has begun. I really hope that you will be able to solve this problem.

No, change is only to reduce overhead of format, not to remove any data from save. 90% of big save is still solder history.

[R1dO]

Aside from compilation time (number of cores seems like a decent explanation).

Any difference in startup time could be explained by different debug profiles (IDE specific).
I suspect Meridian uses an advanced one which shows much more diagnostics (something akin to: https://docs.microsoft.com/en-us/visualstudio/profiling/perftips?view=vs-2022). This requires much more calls (and cache storage) than a simple debug profile that only allows you to break at specific instructions and query the stack.

As for timing numbers: both a full compilation and starting up piratez in debug mode takes ~2minutes on my (may i say .. beefy) computer. This is on vscode (linux snap) where cmake controls the build types.
Then again, I got hit by https://github.com/microsoft/vscode/issues/132025, meaning Meridian would still win if a run involves querying more than 10 variables .

[Ethereal]

No, change is only to reduce overhead of format, not to remove any data from save. 90% of big save is still solder history.

And this is very sad, because to keep the memory of each destroyed unit from the very beginning to the end of the game is a terrible stupidity. And yes, while there are 3-4 hundred of these units, it's unnoticeable. And when there are several thousand of them, the game turns into hell, unless completely abandon "saves\ loads".

[R1dO]

I do believe the following post has the solution if you do not want the stupidity of soldier history:
https://openxcom.org/forum/index.php/topic,2915.msg95102.html#msg95102

[Yankes]

And this is very sad, because to keep the memory of each destroyed unit from the very beginning to the end of the game is a terrible stupidity. And yes, while there are 3-4 hundred of these units, it's unnoticeable. And when there are several thousand of them, the game turns into hell, unless completely abandon "saves\ loads".

Because it change behavior of game, and whole point of this change was do not change behavior.
If you have any concrete suggestions then propose it what should be change and how it will affect current game behavior.

[Ethereal]

I do believe the following post has the solution if you do not want the stupidity of soldier history:
https://openxcom.org/forum/index.php/topic,2915.msg95102.html#msg95102

Thank you for reminding me of my conversation with Meridian four years ago. And after 4 years, everything again comes down to disabling statistics, instead of solving the problem with it. Yes, I will soon forget about her existence! Not to mention the fact that all modifications concerning combat awards lose all meaning.

Because it change behavior of game, and whole point of this change was do not change behavior.
If you have any concrete suggestions then propose it what should be change and how it will affect current game behavior.

https://openxcom.org/forum/index.php/topic,5709.msg87921.html#msg87921

[R1dO]

Well the conclusion from both threads (mine and the one you linked) came down to:
"You can't have your cake and eat it".

Due to the nature of statistics/diaries and how it is used by mods one would have to rewrite a lot of the diaries code in order to save a small amount of text in the saves.
Such a rewrite would be subject to the following limitations:
* Be backwards compatible (older saves)
* Be back portable to openxcom (savegame compatibility)
* Don't break existing mods using the feature
This is a daunting task, which would probably only be undertaken by a dev that is as passionate about file reduction (while keeping the cake) as you are.