Author Topic: Re: [MAIN] XPiratez - N8 28-Apr-2024 Labyrinthus Noctis  (Read 3731169 times)

Offline Callahan

  • Captain
  • ***
  • Posts: 61
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5415 on: March 17, 2018, 04:20:50 pm »
Hi Callahan!
It's not just Ufopaedia; the game simply works all graphic resources to RAM upon startup. Ufopaedia is a small percentage of that, compared to all the sprites, paperdolls etc. Therefore Piratez requires at least 3 GB RAM to play, because it's so huge.

I do not want to say the work to be bad. Not at all. It is an amazing amount of content and detail in it. But the decision to load up all resources upon start is generally one of the least efficient ways to run any programm. For the images and sounds, you will only need a small percentage of that especially early in the game where you lack weapons and database.
Loading stuff if the video/audio pointer is NULL upon use is a good way to load only resources when you actually need them. Except for basic interface items, of course.
ALT + TAB must be a nightmare, too, as all those surfaces have to be restored when coming back.

There are literally hundreds of image and audio files that are probably not even used in the first few game weeks. That being said, I had no chance to run the game, so it's an assumption only.
Maybe one can pursuade the developers to do a bit streamlining? I would really like to play this, but I am waaayyyyy far from having 3 GB of system memory. And future addons will sure continue to take their toll on RAM.

Offline Rince Wind

  • Commander
  • *****
  • Posts: 572
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5416 on: March 17, 2018, 05:46:28 pm »
I don't think this is the correct forum, as I assume this depends on openxcom, and is not something specific to Xpiratez.

And wow, I thought I was playing on a toaster. :D

Offline Callahan

  • Captain
  • ***
  • Posts: 61
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5417 on: March 17, 2018, 07:10:05 pm »
The mod does have it's own executable. So I assume there is a high probability the developers are capable of compiling on their own.
Toaster? Should this imply my PC from 2001 would be, like, old?
Man, I remember times where pixel color info was stored in nibbles. You young hoppers have gone mad about having too much RAM available.

Offline ohartenstein23

  • Commander
  • *****
  • Posts: 1932
  • Flamethrowers fry cyberdisk circuits
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5418 on: March 17, 2018, 07:35:30 pm »
The mod's .exe is developed by Meridian (not Dioxine, the author of the mod), who's forked his work on top of another executable, made by Yankes, who made yet another fork of the original engine.  It still contains all of the features and most of the code of OXC, so in order to prevent merge conflicts on a massive scale, changing how images are loaded into RAM needs to come from the OXC developers, not this particular executable.

Offline Rince Wind

  • Commander
  • *****
  • Posts: 572
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5419 on: March 17, 2018, 07:47:00 pm »
My computer is from 2007 or 2008.

I do remember times when I had to fiddle about in config.sys and autoexec.bat to get enough memory of different kinds for games start.

Offline RSSwizard

  • Commander
  • *****
  • Posts: 762
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5420 on: March 17, 2018, 09:00:58 pm »
I guess the memory use would go down to around 300 to 500 MB.
the game simply works all graphic resources to RAM upon startup.

Ive determined that the real problem is the .OGG music is being converted into some raw format while its being loaded into memory. Its unnecessary even modern-day to do that, and pretty sickening, because various MP3 and video players will decode the OGG in real time because its such a low overhead, far less than displaying video. This is the main stumbling block to being able to get X-Piratez to run under Android due to the huge amount of ram this takes up.

(I figured this out by replacing the music selection with low quality/frequency versions that had a smaller file size, and also truncating some tracks that weren't immediately referenced by the ruleset, and the memory footprint went from about 1 GB to around 750 MB just because of this alone. My main intention doing it was to decrease load times since every startup for x-piratez was taking well over a minute, and it still takes over a minute but its noticeably less now)

Makes me wonder if there is any way to make to load .mid files to run adlib music.

Although its "easier" to decompress everything into memory upon loading its very inefficient, especially since running it under android is a potential target. What you can do is load the resources, and whenever they are referenced as needed, you decompress them into reserved memory. Since there's no heavy lifting involved here, for a mission or loading the ufopedia, all it is is just a small set of images at a time and very short if any loading delay during the game. Besides when you start a mission, it autosaves the map anyway and that automatically incurs a pretty hefty load delay.
(trade 30+seconds initial game load for maybe 1-2 seconds on the front end per new mission)

Toaster? Should this imply my PC from 2001 would be, like, old?
I feel ya man. Ive got 2Gb ram and its not going to increase.

One important note is that they are still using 8bit and locking the game to a palette just like the original. No color translation tables or palette remaps like zdoom does, no 24-bit post-processing color adjustments, no alpha layers or anything like that. So it is pretty true to the old days on that account (maybe painfully...)

I wonder what the native data formats are that they're decoding media into for the images, sounds, and music...

It still contains all of the features and most of the code of OXC, so in order to prevent merge conflicts on a massive scale, changing how images are loaded into RAM needs to come from the OXC developers.

Its a sign of a need to make a change. If not because of the example of X-Piratez getting so large, a big resource overhead probably wasnt anticipated in the early days of openxcom but now its becoming a problem. So a new resource handling module should be on the drawing board.

Forking it wont be an issue if its done right, it can be dropped in and adopted for all versions.
« Last Edit: March 17, 2018, 09:21:39 pm by RSSwizard »

KZad Bhat

  • Guest
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5421 on: March 18, 2018, 08:27:00 am »
I do remember times when I had to fiddle about in config.sys and autoexec.bat to get enough memory of different kinds for games start.

Wizardry 7: Crusaders of the Dark Savant, had to have over 570KB of conventional memory. In MS-DOS 5.0, you couldn't do that with the MSCDEX loaded, so you have to have config.sys and autoexec.bat setups without the CD drive. Love when you also have a very small hard drive, so Wizardry is about all you can get when it's installed.

God, I was glad to find MS-DOS 6.22 with a default memmaker run got me 620KB with the CD-ROM driver loaded!

Offline Callahan

  • Captain
  • ***
  • Posts: 61
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5422 on: March 18, 2018, 11:30:58 am »
Too bad saving memory was no priority from the beginning. I found this to be helpfull with all developments, as you tend to start a programm quite often during the development phase of the engine, as well as during bugfixing. Inefficient memory/resource usage is stressing RAM, hard drive and patience.

Loading on demand and compressed loading are ideas that I consider to be very attractive. There is no gain by storing stuff in RAM that you do not actually use, or storing in an unneccessarily large format.
I guess you are the experts on where to forward these ideas. What would be the right place?

Meanwhile, playing xcom1 & xcom2 will have to suffice.

Ahh, yes, the good old DOS times. Bootdiscs, Highmem.sys, memmaker. Was quite advanced for me when I first got a grip on one of those beauties. I used to operate punch card computing machines before.

niculinux

  • Guest
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5423 on: March 18, 2018, 01:03:31 pm »
The mod's .exe is developed by Meridian (not Dioxine, the author of the mod), who's forked his work on top of another executable, made by Yankes, who made yet another fork of the original engine.  It still contains all of the features and most of the code of OXC, so in order to prevent merge conflicts on a massive scale, changing how images are loaded into RAM needs to come from the OXC developers, not this particular executable.

I think it is a wanted decision (preloading all in RAM) to avoid slowdowns during the game, and all id really quick responsive ☺ maybe something may be  done with the musuc as RSSWizard pointed out so better ask developers in the right forum ☺☺

Offline Ragshak

  • Colonel
  • ****
  • Posts: 136
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5424 on: March 18, 2018, 02:08:23 pm »
Maybe all this is the reason why sometimes I cant launch X-Piratez on my notebook after working some time. I need to restart.

Offline Meridian

  • Global Moderator
  • Commander
  • *****
  • Posts: 8671
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5425 on: March 18, 2018, 03:06:52 pm »
Too bad saving memory was no priority from the beginning.

The project started as re-creation of the original xcom, which fits on a few floppy disks.

Nobody expected mods 50-times bigger than the original. (I would even dare to say nobody expected mods at all at the very beginning)
Actually, considering original resources are not compressed (much) and mod resources are mostly compressed GIFs and PNGs, 50-times is an understatement. Probably more like 100-200x bigger, if not more.

Vanilla is eating up about 40 MB of RAM... including everything, not only resources (which are probably just 5-10 megs).
I guess you can't blame anyone of loading 40 MB into RAM in 2018.

PirateZ eats up about 1000 MB of RAM... far from ideal... but it does still run on 1.5+ GB Android devices.
(Also, just open a Task Manager and look at your Firefox/Chrome... there's a very high chance they are eating more RAM than PirateZ)

If anyone has such a problem with that... we are open source, feel free to fork and change it.

Offline Callahan

  • Captain
  • ***
  • Posts: 61
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5426 on: March 18, 2018, 04:41:54 pm »
I do not want to "blame" anyone for the game using too much memory. It is just sad it is the way it is because it does not work for me.
Probably, I am always trying to reduce RAM usage because I am from a time where every byte did matter.
I even think about every variable if it must be be INT16 or if INT8 would do it. A habbit hard to get rid of, even when I know this does not matter much today.

This pirate mod made my system pass out at 1.5 GB RAM used, as I only have 512 MB. I also read that it would consume up to 3 GB.
I'd really be happy if you could rework your version to be more friendly toward huge mod contents. (and keep it running under 32bit OS)
Since you are the person most familiar with the code, you would be the best choice.

Offline RSSwizard

  • Commander
  • *****
  • Posts: 762
    • View Profile
Re: Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5427 on: March 18, 2018, 05:27:45 pm »
I do not want to "blame" anyone for the game using too much memory. It is just sad it is the way it is because it does not work for me.
Im going to package up some low quality .ogg file replacements for the game so that anyone who has this problem might be able to play or reduce the stress on their ram. Might take me a couple days. Keep in mind that low quality is noticeable, and to make it effective you'd have to first move all those music files somewhere else (also there will be some switched tracks, call it a mod if you want, im a bull in a china shop like that).

Offline Rince Wind

  • Commander
  • *****
  • Posts: 572
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5428 on: March 18, 2018, 05:36:59 pm »
Would it be possible to just delete the music? I have it turned off anyway.

Offline RSSwizard

  • Commander
  • *****
  • Posts: 762
    • View Profile
Re: [MAIN] XPiratez - 0.99I1 - 26 Dec - Spirit of Christmas
« Reply #5429 on: March 18, 2018, 06:05:08 pm »
Would it be possible to just delete the music? I have it turned off anyway.
I doubt it, but if you want to take the suicide lane on this, make a new folder and copy the lowest filesize file you find in the folder (I think its GMENBASE, the music that plays during a mission briefing). Copy it as many times as there are files in the music folder, then rename them to each of the listed ones. Among the GEO series I think the only one which is directly referenced is #3, used during the intro sequence, so any of them beyond #3 can probably be deleted flat out (#1 and #2 should probably be there just in case). Then delete/move-backup all the music in the SOUND folder, and copy your dummy copies into it.


Edit:
Here is the modified music package, which also includes some fallout musics for battlescape. It only weighs in at about 21Mb. Instructions are included.
https://mega.nz/#!iogH1AKb!09I1SbCp_Jl8u8ZwCmiAXEVLqAdtKUzRTPBvpuKcvvI
« Last Edit: March 18, 2018, 06:39:09 pm by RSSwizard »