Author Topic: Reversing to Orion - project 1oom  (Read 6990 times)

Offline Kilgore T.M. Replicant

  • Sergeant
  • **
  • Posts: 76
    • View Profile
Re: Reversing to Orion - project 1oom
« Reply #135 on: October 02, 2018, 02:24:13 pm »
1oom v0.9 has been released. Changes:
    - map scroll speed is adjustable
    - Planets and Fleet screen lists sortable by name/count/bases/etc
    - keys!
        * Alt-X in main menu toggles -uiextra
        * UHJK scroll the map
        * / to search for a planet by name
        * (Ctrl-)1-9 to tag/recall planet
        * space bar for Continue/dismiss/OK/etc
        * 1-5 as portrait click on Races
    - almost 20 bugs fixed, notably:
        * a cause for the infamous 32000 doom stacks indentified and fixed
        * AI scrapped wrong ship designs resulting in obsolete fleets
        * AI II armor bias fixed, resulting in overall better ship designs
        * all but a few of the 1.40m fixes are now implemented
    - doc/pbxin_fixbugs.txt expanded, including fix for waste calculation

Source code: https://gitlab.com/KilgoreTroutMaskReplicant/1oom
Binaries: https://gitlab.com/KilgoreTroutMaskReplicant/1oom/tags/v0.9
Homepage: https://kilgoretroutmaskreplicant.gitlab.io/plain-html/

Test. Report. Profit?

Offline Kilgore T.M. Replicant

  • Sergeant
  • **
  • Posts: 76
    • View Profile
Re: Reversing to Orion - project 1oom
« Reply #136 on: October 14, 2018, 11:16:23 am »
Time for a rantrospective.

Over a year ago I stumbled on a MOO1 Le'ts Play [sic] and got bitten hard by nostalgia. On 2017-10-14 I went looking for a Free Software Master of Orion engine to play with. Finding none, I said "fuck it" and rolled up my sleeves. One year later, the LP is wrapping up with a bonus round and we have 1oom going the last inches towards a stable v1.0 release.

Not finding any ready made disassembly was a disappointment. Documentation on the save format gave a starting point.

The disassembly was started using radare2 which turned out inadequate. Finding no better Free tool, I ate my principles and installed wine + IDA 5.0 freeware. It really is a shame that no Free tool is up to the task.

Going from nothing to v0.1 took 6 months of disassembling, manual decompiling to C and refactoring. During those months I worked about 9/7 on average.

The human unautomated part of disassembling is figuring out what "func_1A562" which operates on variables "var_2" and "unk_39BD8" does and naming all of that to something sensible. Here the knowledge on save format allowed to name many variables after spotting the load/save code.

Decompiling is the process of translating assembly to C. As an example, let's take a peek at one C function:
Code: [Select]
void game_planet_destroy(struct game_s *g, uint8_t planet_i)
{
    planet_t *p = &(g->planet[planet_i]);
    /* ... */
    p->rebels = 0;
    p->unrest = 0;
    p->reserve = 0;
    p->prev_owner = p->owner;
    /* ... */
}
And what it looks like in the disassembled v1.3 EXE:
Code: [Select]
game_planet_destroy proc far

var_2           = word ptr -2
planet_i        = word ptr  6

    push    bp
    mov     bp, sp
    sub     sp, 2
    push    si
    push    di
    mov     cx, [bp+planet_i]
    ...
    mov     ax, cx
    mov     dx, size struc_planet
    imul    dx
    les     bx, planet_data_segoffs
    add     bx, ax
    mov     es:[bx+struc_planet.rebels], 0
    mov     ax, cx
    mov     dx, size struc_planet
    imul    dx
    les     bx, planet_data_segoffs
    add     bx, ax
    mov     es:[bx+struc_planet.unrest], 0
    mov     ax, cx
    mov     dx, size struc_planet
    imul    dx
    les     bx, planet_data_segoffs
    add     bx, ax
    mov     word ptr es:[bx+(struc_planet.reserve+2)], 0
    mov     word ptr es:[bx+struc_planet.reserve], 0
    mov     ax, cx
    mov     dx, size struc_planet
    imul    dx
    les     bx, planet_data_segoffs
    add     bx, ax
    mov     ax, es:[bx+struc_planet.owner]
    push    ax
    mov     ax, cx
    mov     dx, size struc_planet
    imul    dx
    les     bx, planet_data_segoffs
    add     bx, ax
    pop     ax
    mov     es:[bx+struc_planet.prev_owner], ax
    ...
That's right: every time a planet (or, well, anything) is read or written to, the address is recalculated. The EXEs could be shrunk down to less than 50% size. Picture wading through about 770 kB of that garbage and that's my life for most of the 6 months before v0.1.

Refactoring consisted of getting rid of global variables and splitting game/UI/AI code allowing switchable AIs, multiplayer and atrocities like the cmdline UI. The game/UI boundary violation that most stuck to the mind was Plasma Torpedo depletion being handled in missile drawing code.

I needed a new nick before going public with this. Why use one unpopular culture reference when you can chain several? Answer: username length limits.

With 1oom-0.1.tar.bz2 in hand I embarked on a journey to publish it. The first place I tried was the home of the unofficial patch: Realms Beyond. Unfortunately they block IPs of "known spammers". After many attempts at fighting against the CAPTCHA to get blocked and seeing the Contact form do nothing, I set my sights on the FreeOrion forum only for the topic to be rejected for containing "links to illegal or pirated software". Registration attempts at the official corpse-of-Simtex-on-display site masteroforion.com were a futile exercise in staring at a spinny please wait circle. Next up was OpenXcom forums (sci-fi, MicroProse, 1993, GPL == close enough). Fresh out of ideas and waiting for the OXC registration to be sorted out, I went to Reddit of all places to make the first public announcement. This thread followed soon after.

It took about 2 weeks to get the first tarball out. That's a lot of lost productivity.

Suggestions to put the project in GitHub followed. I obliged. Then Microsoft bought GitHub. Hello GitLab!

Once a foothold was established in Realms Beyond, the good ideas started flooding in. The feature creep has going on since then. So much for simply replicating v1.3.

The bugs have never piled up. The project has always been starving for bug reports. These days I work on this for about 2/7, mostly towards network multiplayer.

The current status of the project is a fully functional engine replacement with less bugs (I hope) and nifty new features. I see no reason for anyone to use the old DOS EXEs anymore. Mission accomplished?

How to calculate the value of an entertainment product when money is taken out of the equation? I propose comparing the time spent on developing the product to the total time spent on consuming it. I suspect 1oom has not broke even yet.

Cheers to:
- sargon0 for the partial save format docs
- SupSuper for tolerating this thread and helping out with the Windows port
- Stoddard for the invaluable devbuilds and SDL2 fixes
- Hythlodaeus for getting a foot in the Realms Beyond door
- coder111 for the planetary governor
- Jeff Graw for the UI ideas and feedback
- RefSteel for MOO wisdom and forum archaeology
- all who cared enough to report a bug

Lines of code in CSV format:
Code: [Select]
date,version,total,src,game,classic,cmdline,sdl,sdl1,sdl2,alleg,alleg4,dos,win,unix
2017-10-14,vnone,0,0,0,0,0,0,0,0,0,0,0,0,0
2017-11-14,v0.0,2568,896,1641,0,0,0,31,0,0,0,0,0,0
2017-12-14,v0.0-39-gfa00f4b,18204,6046,4437,5742,436,822,490,0,0,0,0,84,147
2018-01-14,v0.0-68-geb03567,30847,6430,8729,13628,436,871,522,0,0,0,0,84,147
2018-02-14,v0.0-96-ge0d2c60,39674,7821,13539,16124,537,878,544,0,0,0,0,84,147
2018-03-14,v0.0-117-g62059b7,49070,8692,18103,19964,633,878,544,0,0,0,0,90,166
2018-04-14,v0.1,57149,9359,23802,21570,723,900,539,0,0,0,0,90,166
2018-05-14,v0.2-55-g681cf86,62852,11419,24160,21694,2816,808,751,908,0,0,0,121,175
2018-06-14,v0.4-38-g5a9834d,67158,12274,25015,22068,3349,1075,816,987,648,453,148,137,188
2018-07-14,v0.5-67-gbe8d753,70168,12438,26229,22477,4216,1196,951,1063,662,463,148,137,188
2018-08-14,v0.6-46-g1ca378e,74222,13502,26696,22794,6315,1209,982,1066,675,467,162,152,202
2018-09-14,v0.8-14-g94ab478,74707,13534,27055,22890,6279,1209,995,1087,675,467,162,152,202
2018-10-14,v0.9-10-gdb25388,75508,13537,27143,23597,6284,1201,994,1094,675,467,162,152,202

Misc:
- hardest bug to fix: the DOS mouse crash
- longest bug post-mortem: #46
- most head-to-desk function: src/gfxaux.c:gfx_aux_draw_frame_from_rotate_limit_do
- stupidest mistake: not reading the Official Strategy Guide before starting
- worst idea: no additional files
- games played since starting the project: 3 (1 to completion)

TL;DR dear diary, while waiting for bug reports I disappeared up my own ass

Offline Stoddard

  • Captain
  • ***
  • Posts: 341
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: Reversing to Orion - project 1oom
« Reply #137 on: October 14, 2018, 02:27:22 pm »
This is a magnificent work.

Congratulations!