Battlescape development

[pmprog]

I've just seen a push for Mars battles. Is that the game completable?

[Warboy1982]

it is now.

[pmprog]

Awesome! Have to get set up to start my Interactive Let's Play

[Daiky]

It seems some people are saving their game during battle, I can advice to save and load as much as possible during geoscape only, and try to be a man and don't save/load during battle :p

But anyhow, for those people that still insist to save during battle, I'm (again) (thinking of) saving the battlescape terrain in a binary way.
Because of a current battlescape savegame, that's let's say 18000 lines, there are 12000 lines for the terrain only. It's a bit crazy to have it described in plain text. It's a waste of CPU time and Harddisk space and serves no real purpose, as it's almost as difficult to manually edit as it would be a binary blob.

So I've decided on the width of the fields, they are unsigned and as follows: 32bit for the index, (8bits for the terrain ID + 16 bits for the map data id ) x 4 terrain objects, 8 bits for smoke, 8 bits for fire, 8 bits for discovered flag (room for 4 player and 4 alien discovered flags, if that ever might be of any use)

On top of that the "nodelink" concept will be thrown in the bin. The distance between nodes and whether or not they can be reached are calculated on the fly - the precalculated values are nowhere used, but they take up 1500 lines in the savegame file and a bunch of useless objects in memory.

But in the mean time, save/load as much as possible in geoscape only 

[pmprog]

I save when my episode is over - maybe I should just make 3 parters if necessary 

I do agree that load/save times really need to be improved, so anything you can do would be great

[SupSuper]

Did you see the previous discussions regarding load/save? They had some interesting ideas (eg. only save non-default values, don't save blanks/nulls/empty/etc):

https://openxcom.org/forum/index.php/topic,799.0.html
https://openxcom.org/forum/index.php/topic,758.15.html

Also I've been informed that the newest yaml-cpp version is incompatible with OpenXcom as they've changed to a new better API. For now we can stick to the old one, but at some point we'll have to rewrite it all... I can already see the sheer excitement in your face.

[Daiky]

Alienfoods' code change has been already implemented by Warboy in the master branch. My savegame with alienfoods' improvement is now 422kb and using my binary method further reduced to 161kb + the save/load time is reduced even more, because instead of a yaml key/value pair for each tile, the binary blob with all tiles is just 1 key/value pair.

[Volutar]

Very nice. Having map(tile) data in text format was insane. Daiky, I suppose unit/object/route data is still text? And map data is some single "huge blob" encoded in something like UUE or base64?

[Daiky]

The rest is still text, but they are a lot smaller. The yaml library automatically encodes binary to base64.

edit: having lots of troubles getting it working however. I'm not very experienced with binary, for example how to convert a series of unsigned char * to int's or vice versa. I'm doing things like buffer[ptr] + buffer[ptr+1] << 8 + buffer[ptr+2] << 16 + buffer[ptr+3] << 24 ... don't know if it's best practice. The version I currently have is constantly crashing on loading, so there must be things wrong.

[Yankes]

The rest is still text, but they are a lot smaller. The yaml library automatically encodes binary to base64.

edit: having lots of troubles getting it working however. I'm not very experienced with binary, for example how to convert a series of unsigned char * to int's or vice versa. I'm doing things like buffer[ptr] + buffer[ptr+1] << 8 + buffer[ptr+2] << 16 + buffer[ptr+3] << 24 ... don't know if it's best practice. The version I currently have is constantly crashing on loading, so there must be things wrong.

try this

Code: [Select]

buffer[ptr] + (buffer[ptr+1] << 8) + (buffer[ptr+2] << 16) + (buffer[ptr+3] << 24)and remember

Code: [Select]

std::cout<< a + 5 << std::endl;use same operator :>

[hmaon]

try this

Code: [Select]

buffer[ptr] + (buffer[ptr+1] << 8) + (buffer[ptr+2] << 16) + (buffer[ptr+3] << 24)

try also

Code: [Select]

*((int*) (buffer+ptr))or

Code: [Select]

*((int*) &(buffer[ptr]))
That is, assuming the number is always in native endianness. The code with the shifts and adds assumes a little-endian integer.

edit: Also, if you use the shifts and adds code, make sure buffer is declared unsigned char*. Otherwise you will have trouble.

Test code:

Code: [Select]

#include<stdio.h>
#include<stdlib.h>

int main()
{
    unsigned char buffer[] = {0xff, 0x7f, 0, 0};
    char caveat[] = {0xff, 0x7f, 0, 0};
    int ptr = 0;

    printf("%d\n", *((int*) (buffer+ptr)));
    printf("%d\n", buffer[ptr] + (buffer[ptr+1] << 8) + (buffer[ptr+2] << 16) + (buffer[ptr+3] << 24));
    printf("%d\n", caveat[ptr] + (caveat[ptr+1] << 8) + (caveat[ptr+2] << 16) + (caveat[ptr+3] << 24));
}

output:

Code: [Select]

32767
32767
32511


[Daiky]

thanks for the hint.
So if you save a file and transfer it to a system with other endianness and try to load it there, that'll not work with that casting/pointer thing. So I prefer the one with the shifts, so you convert using same endianness?

[hmaon]

thanks for the hint.
So if you save a file and transfer it to a system with other endianness and try to load it there, that'll not work with that casting/pointer thing. So I prefer the one with the shifts, so you convert using same endianness?

Yeah. Just sure you're writing the integers for saving also using shifts and in the same order, of course.

Another way to make code endianness agnostic is with the htonl() and ntohl() functions, which may be faster and easier to read. Network byte order is big-endian but the byte swap can be implemented using one bswap instruction on x86 instead of four shifts. Also, the one 32-bit read from memory is likely to be faster than reading 8 bytes individually.

So,

Code: [Select]

i = ntohl(*((int*) (buffer+ptr)));
and the write operation:

Code: [Select]

*((int*) (buffer+ptr)) = htonl(i);

It's not a big deal though, I guess.

[SupSuper]

Btw SDL has built-in functions to deal with endianness: https://www.libsdl.org/intro.en/usingendian.html

[Daiky]

I've put this little project on hold until we move to the new yaml cpp API I think, because I fail to see how the loading from binary nodes automatically base64 decodes using the old API. The saving (and base64 encoding) works fine, but the at loading I'm stuck.