[OLD] Old OXCE Future plans

[Stoddard]

I will need lot of refactoring to make it work (every function that load resources need be changed to allow loading from zip).

Plan is to change them all to accept streambufs and hand out them from the file map.

Some function will simply not work because they use function that allow only file paths to real files.

So far I haven't seen anyhing like that in the code.

Another thing is finding correct library that will work with GCC and VS.

Oh I'm sure miniz would be fine.

[Yankes]

I look around code and I see that it will require some refactor but not that much.

@Stoddard

Code: [Select]

IMG_Load(utf8.c_str())
How do you handle code like that? But luckily this lib have good api and allow load from arbitrary array: `IMG_Load_RW` and `SDL_RWFromMem`.
We dodged bullet there

Overall I think that in most cases function `FileMap::getFilePath` should be replaced by `getFile` that return this streambuffer or even simpler `std::vector` in struct with some helper methods.

[KZad Bhat]

In case anyone has missed it, allowing resources to load from an archive file also cuts down on the size of the installed mod significantly, even if you don't use compression. After all, probably cutting down on disk space used by an average 1.5KB . . . times 30k+ files. Okay, 29k+ files, some chunks of what's in there should not and even cannot be archived. But damn what a savings for people who have to think of storage space! Can't say how much this cuts down on the size of the fully wrapped .zip for download, but there could be significant savings there, too.

[yergnoor]

Can't say how much this cuts down on the size of the fully wrapped .zip for download, but there could be significant savings there, too.

Quite a few

Whereas the X-Piratez mod, with a total weight of 229 megabytes, takes 760 megabytes because of the peculiarities of the file system. If you save it in zip, even without compression, you will need half a gigabyte less space.

Depends on the file system, the example I gave when using Fat32, all files are 19623 in the latest version of X-Piratez.

[Stoddard]

The refactoring (what a name for a horror movie) got a bit out of hand.

I decided for now to just use SDL_RWops for the buffers.

std::vectors are awkward since file format parsers need to seek and/or incrementally read,
while std::streambufs are overweight and in any case 90% of ops are IMG_Load_RW which only ends up converting the streambuf to SDL_RWops.

[Yankes]

But who will maintain lifetime of this `SDL_RWops`? Bes would be if this would be done automatically.

Overall when I say `std::vector` I mean only use it as storage not as API.

[Stoddard]

But who will maintain lifetime of this `SDL_RWops`? Bes would be if this would be done automatically.

I don't see any difference between stdlib and SDL objects in this case. Once you're done, you just call either delete or SDL_FreeRW() and that's it.

[Yankes]

Once you're done, you just call either delete or SDL_FreeRW() and that's it.

That was my point do not call explicitly any `delete` or `SDL_FreeRW`, it should be done automatically when data go outside scope (RAII idiom).
Right now OXCE use C++11 and in it you have enough tools to not use any explicit `delete` in code. You can return vector by value and move it to load function and it will be not slower than using pointers.

Example is my WIP commit: https://github.com/Yankes/OpenXcom/commit/a87fdcb2623b02e7797b0734970c017e5a533f70#diff-2f13ead716b9567315c18615865bcd0bR50
where I remove all `delete` and `new` from surface set.

[Stoddard]

That was my point do not call explicitly any `delete` or `SDL_FreeRW`, it should be done automatically when data go outside scope (RAII idiom).

Yeah, I know that. It has its uses, but in my opinion only complicates things in this specific situation.

Specific situation being loading graphics and parsing data in vanilla file formats.

Non-vanilla graphics do require SDL_RWops. Vanilla graphics - well, reading bytes from streams/vectors and then calling SDL's byteswappers on that is much less readable than just calling SDL_ReadLE16() and friends. Ditto for the rest of vanilla file formats.

For rulesets we can as well have another interface to the file map, but I haven't got to that yet.

I'm all for code simplicity and readability, and in this specific case my opinion is that using C++ data types is counterproductive.


For the Surfaces and SurfaceSets I actually did something pretty much in line with your commit - have a bunch of Surface constructors instead of Surface::loadWhatever()s , get rid of resize(), etc.

no commits yet though, too much might still change

[RareOne]

Soooo, sorry, don't want to sound rude.
But how's it going?

Too much work for small unnecessary result?

[Meridian]

Soooo, sorry, don't want to sound rude.
But how's it going?

Too much work for small unnecessary result?

If you're asking about loading from zip files... it's in progress.

It's a big task, so don't expect it to be ready sooner than 2+ months from now.

[RareOne]

Wow, thanks for the reply.
I assumed it would take a large amount of time.
It's just that for some reason I had the impression from the first Stoddard's reply that it would take only a weekend to implement that change.
I hope it all works out for the best of everyone!

[Meridian]

Yeah, easy (not hard) is not the same as quick (not time consuming).

Counting to a billion is not hard... but it will take you about 31 years, give or take

[RareOne]

I understand that, I just hoped there could be some kind of script to automatize the changes. Woe silly me.
I guess it's true that programmer's dream is to programm the process of programming. 

But I've come up with a couple new requests.
Is it possible to force game cursor show upon/replace the system one. I plugged a mouse to the phone, see...
And is it possible to add a button in-game or to the system tray along back, menu and so on buttons that would call the keyboard any time, not only when you type smth? Like, to use quick save and alternate monement methods?

[Stoddard]

For an update - well, yeah I got most everything except rulesets, PCK/TABs and music/sounds loading from not-plain-files.
It's music/sounds that are giving me trouble. I expect PCK/TABs working this evening, rulesets are even more trivial. Audio.. it's idk maybe the oldest part of the code. Or seems to be. So a lot of work fo the refactorhammer.

Could've been faster, but for ease of debugging I went and made android branch compile and work on desktop linux without the  sfalexrog's outdated forks of SDL_everything. This way I at least can test the code as it would run.

But I've come up with a couple new requests.
Is it possible to force game cursor show upon/replace the system one. I plugged a mouse to the phone, see...

I'm sure it's possible. After something I did the cursor suddenly appeared in all its triangular glory. So sure.
Although it'll take some time to hide it back.

And is it possible to add a button in-game or to the system tray along back, menu and so on buttons that would call the keyboard any time, not only when you type smth? Like, to use quick save and alternate monement methods?

This I have no idea how to do, but noted.

Let me first make the fork at least not crash on my phone.
Sadly I only have this "honor 9l" excuse for a phone on me right now, having drowned the nexus 6P in a river recently. So the testing is hampered.


UPDATE 21 Oct:
Managed to get rid of every single one FileMap::getFilePath() call, everything is now loaded via SDL_RWops and std::istreams handed out of FileMap. Now on to fixing all the bugs that crawled out - and there are many. For example the one with android version crashing on pressing Options in the main menu - I almost got it, but it managed to hide somewhere.