Author Topic: MAPVIEW upgrade  (Read 259777 times)

Offline hellrazor

  • Commander
  • *****
  • Posts: 2013
  • Deep Ruleset Digger & Bughunter
    • View Profile
    • Github Account
Re: MAPVIEW upgrade
« Reply #360 on: January 27, 2019, 05:07:27 pm »
Any way to easily convert my old MapView settings into the new format?

See attached files.

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: MAPVIEW upgrade
« Reply #361 on: January 27, 2019, 05:28:15 pm »
The CPU load gets pretty high thou. Around 30% constantly on my X220.

Same here on phenom2 975. It isn't easy on the cpu on windows either. No easy way around it alas.


Quote
Also i experienced some crashes, when trying to resize/recontainer the seperate windows with i3 windows manager.
Rescale and redraw routine seems not so good....

Also any decent keyboard shortcuts avaible?

I have no idea.. report them here I guess.

Quote
In any case thank you all for enabling me to create maps natively under Linux :)
Let the map making maze begin. As soon as is have enough time thou ;)

Thank you. I'm glad something good came out of this effort.
« Last Edit: January 27, 2019, 05:50:48 pm by Stoddard »

Offline kevL

  • Colonel
  • ****
  • Posts: 466
  • pitchforks and torches
    • View Profile
Re: MAPVIEW upgrade
« Reply #362 on: January 27, 2019, 10:26:09 pm »
The CPU load gets pretty high thou. Around 30% constantly on my X220.

Also i experienced some crashes, when trying to resize/recontainer the seperate windows with i3 windows manager.
Rescale and redraw routine seems not so good....

have to wait for a Linux c#/.net programmer to have a look ... and, y, it's not a highperformance app ( I get the impression that .net is not the way to go for HP. Am sure that a coder w/ more experience than me would have an assortment of so-called tricks to use, though )

Quote
Also any decent keyboard shortcuts avaible?

not really. They're all mentioned in the CHM help ...


Any way to easily convert my old MapView settings into the new format?

unfortunately, ConfigConverter handles only simple/straightforward MapEdit.DAT files. Terrains/Images in multiple directories is a no-go. I might try to re-code the converter to handle Images.Dat at some pt

i can only suggest setting up your tilesets in smaller chunks (as you need them) for now. That is, avoid the tempation of "I want it all. now" if you know what i mean,


/cheers

ps. If you need to setup Maps with terrains in multiple folders (as I believe the OxC mod-system works) wait just a day or so. I'll be releasing a build that supports terrains from multiple folders soon here:

https://github.com/kevL/OpenXCOM.Tools/blob/master/Distribution/README.md

(not sure when Stoddard's linux-build will pick it up tho)


EDIT__
next: Codename Luke ;)

- Option to embolden every 10th gridline (color & width)
- print current selection size x/y in statusbar of MainView
- Routes Editmenu: nodeup/nodedown 1 level

- working on MapEdit/Images.dat converter ... could take a couple days, no guarantees but looks feasible. (note, a restriction of Mv2 is that MAPS and ROUTES must be sibling dirs)
« Last Edit: January 29, 2019, 12:48:01 am by kevL »

Offline hellrazor

  • Commander
  • *****
  • Posts: 2013
  • Deep Ruleset Digger & Bughunter
    • View Profile
    • Github Account
Re: MAPVIEW upgrade
« Reply #363 on: January 29, 2019, 09:56:04 am »
have to wait for a Linux c#/.net programmer to have a look ... and, y, it's not a highperformance app ( I get the impression that .net is not the way to go for HP. Am sure that a coder w/ more experience than me would have an assortment of so-called tricks to use, though )

Jeah its not such a big issue for the time being. Performance optimisation always comes last. More important is to make it functional, with the features we need and want, Then bugfree (mostly), then performance can be somehow achieved.

not really. They're all mentioned in the CHM help ...

Blind me... Maybe something more traditional like README.txt or KEYBOARDSHORTCUTS.txt or so?

unfortunately, ConfigConverter handles only simple/straightforward MapEdit.DAT files. Terrains/Images in multiple directories is a no-go. I might try to re-code the converter to handle Images.Dat at some pt

i can only suggest setting up your tilesets in smaller chunks (as you need them) for now. That is, avoid the tempation of "I want it all. now" if you know what i mean,


/cheers

ps. If you need to setup Maps with terrains in multiple folders (as I believe the OxC mod-system works) wait just a day or so. I'll be releasing a build that supports terrains from multiple folders soon here:

https://github.com/kevL/OpenXCOM.Tools/blob/master/Distribution/README.md

(not sure when Stoddard's linux-build will pick it up tho)

Each mod has its own directories for MAPS, ROUTES and TERRAIN if the modder so desires.
MAPS is for *.MAP files, ROUTES for *.RMP. Each MAP file needs to have a corresponding RMP file.
Also the directory names are hardcoded to uppercase, if i recall correctly.

The directory TERRAIN, includes the mods own created or modified *.MCD, *.PCK and *.TAB files. Each set here forms one useable terrain set (limited to 256 per file (and or map), if i recall right).

It would be very much appreciated if we are simply allowed to hook, TERRAIN files towards the respective MAP/RMP configuration, which we desire.


EDIT__
next: Codename Luke ;)

- Option to embolden every 10th gridline (color & width)
- print current selection size x/y in statusbar of MainView
- Routes Editmenu: nodeup/nodedown 1 level

- working on MapEdit/Images.dat converter ... could take a couple days, no guarantees but looks feasible. (note, a restriction of Mv2 is that MAPS and ROUTES must be sibling dirs)

Looking forward to the gridlinecolor, for bigger maps. Maybe even tailor this down to 5?

Thank you so far for the great work and your time!

Offline luke83

  • Commander
  • *****
  • Posts: 1558
    • View Profile
    • openxcommods
Re: MAPVIEW upgrade
« Reply #364 on: January 29, 2019, 10:16:19 am »
OK, i can recreate this error with Routes jumping to wrong level
Open a existing map
change the Level ( i  changed a 1 level map to a 2 level map), i
f you check in the Routeview screen it looks correct.
Switch to another map and when it asks, tell it to save changes
Switch back to map you changed, the Routes are now on wrong level.

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: MAPVIEW upgrade
« Reply #365 on: January 29, 2019, 02:39:42 pm »
(not sure when Stoddard's linux-build will pick it up tho)

Builds track the master branch on https://github.com/kevL/OpenXCOM.Tools.git

It's checked at least once an hour.

« Last Edit: January 30, 2019, 12:29:16 pm by Stoddard »

Offline kevL

  • Colonel
  • ****
  • Posts: 466
  • pitchforks and torches
    • View Profile
Re: MAPVIEW upgrade
« Reply #366 on: January 30, 2019, 12:06:24 am »
Each mod has its own directories for MAPS, ROUTES and TERRAIN if the modder so desires.
MAPS is for *.MAP files, ROUTES for *.RMP. Each MAP file needs to have a corresponding RMP file.
Also the directory names are hardcoded to uppercase, if i recall correctly.

sounds right. Mv2's recent change to non-windows compatibility -- ie, file system case-sensitivity -- lends some awkwardness. Eg, the file-labels in MapTilesets used to be allcaps regardless of case on the HD. But since case is now respected, it's possible to add terrain "ufo1" to a Map that is already shown using "UFO1" ...

as long as we all keep our heads screwed on it should be alright

to that end, Suggest that all directories be uppercase (MAPS,ROUTES,TERRAIN) and all files in them uppercase also.

( and.. ASCII is good )


Quote
The directory TERRAIN, includes the mods own created or modified *.MCD, *.PCK and *.TAB files. Each set here forms one useable terrain set (limited to 256 per file (and or map), if i recall right).

i was actually looking deeper into this the other day. The maximum terrain-ID is 253 in a mapfile: the value is stored in a byte (256) but when written to/read from the Mapfile they are offset by 2. So, 254 (0..253)

Quote
https://www.ufopaedia.org/index.php/MAPS

The reason for this is that, within the separate MAP files, a value of 2 maps to the index 0 within the associated MCD set (the first tile). This is as opposed to MAP.DAT, where a value of 2 maps directly to index 2 in the collated MCD array (the third tile). Values of 0 always mean "no tile" (except at ground level, where tile 1 of BLANKS.MCD will be used for to stop mysterious holes appearing in the bottom of the the battlescape), and tile 1 of BLANKS.MCD (burnt earth) is available for use by all tilesets.


qotd: "Nobody needs more than 640k ram"


I recently changed the Mapfile write code to cap assigned IDs to 253. Assigned tileparts that exceed that ID get replaced by a null-part. A warning is issued either when the Map loads or on returning from the TilesetEditor (but not on Map save). It's a warning and not an error, because it seems okay to have tilepart IDs that exceed the limit -- as long as they are not placed on the Map.



Quote
It would be very much appreciated if we are simply allowed to hook, TERRAIN files towards the respective MAP/RMP configuration, which we desire.

on the urging of davide this has been implemented (recently).

In Mv2 the def'n of a "basepath" is "the parent of a MAPS, ROUTES, or TERRAIN folder". Files for .MAP have to be in MAPS, for .RMP have to be in ROUTES (but only a sibling of the Map's MAPS directory), and for .MCD/PCK/TAB have to be in TERRAIN. The Configurator needs a basepath. The Map does not need a basepath if it's in the Configurator's basepath (but each Map can be set to its own basepath). Terrains can use the basepath of the Configurator, or the basepath of their Map, or be read from their own (specified) basepath individually. In effect any terrain can now be anywhere, as long as its in a TERRAIN directory.


Quote
Looking forward to the gridlinecolor, for bigger maps. Maybe even tailor this down to 5?

am gonna try it on 10 for a while... the grids are beginning to look a bit busy atm...


ConfigConverter2 is coming along. It requires MapEdit.dat, Images.dat, and Paths.pth and outputs MapTilesets.yml ... i hardly expect it to work BUT IT MIGHT /luL




Builds track the master branch on https://github.com/kevL/OpenXCOM.Tools.git

It's checked once an hour at most.

thankie



@luke83 - will investigate soon(ish)
I couldn't get it to bork by following (my interpretation of) your directions, but there's still some fishy stuff going on ... will vet through the mapresize code and see what happens.
REPLICATED
RoutesChanged = true; should fix it ... doh.


2019 jan 30
the changes mentioned in my previous coupla posts have been pushed to Master and should be available via Stoddard's linux-build anytime,
I'll release a windows-build tomorrow ... today

https://github.com/kevL/OpenXCOM.Tools/blob/master/Distribution/README.md

2019 feb 4 - update

2019 feb 5
- fix for first run: the initial Configuration threw a KeyNotFoundException exception (by trying to show the current, nonexistent, configuration paths in the Configurator's textboxes)
- etc

2019 feb 7
- RouteView: export/import .RMP files (ie, use .RMP files as templates and/or replace route-nodes of a Map with nodes from a different Map, see RouteView's File menu)
- etc
« Last Edit: February 08, 2019, 08:34:30 am by kevL »

Offline hellrazor

  • Commander
  • *****
  • Posts: 2013
  • Deep Ruleset Digger & Bughunter
    • View Profile
    • Github Account
Re: MAPVIEW upgrade
« Reply #367 on: February 09, 2019, 05:01:47 pm »
I am wondering if i am getting into a issue with the following MCD file group. Since the asigned PCK's are over 256.
See Screenshot attached.

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: MAPVIEW upgrade
« Reply #368 on: February 09, 2019, 05:22:15 pm »
Let's maybe design a map format without that 256 MCD limit?

I propose a YAML structure like

Code: [Select]
- name: THEMAPNAME
    size: [ x, y, z ]
    terrains:
        - BLANK
        - etc
        - etc
          ...
    maybeExpectedTerrainSetSizes:
        - 2
        - 23
          ...
    routes:
     - id: arbitrary string
       posn: [ x, y, z ]
       links:
         - target: {id|N|W|E|S}
           distance: <int> # is used as weight?
           unit: {any|flying|flying_large|large|small}
         - target: ...
           ...
    tiles: !binary |
        # deflate-compressed (gzip header present),
        # base64 encoded and multiline - only to keep text editors happier
        # x*y*z of
        # { uint32_t floor, west, north, object }[]
        # in the same order as is .MAP :
        # left to right, top to bottom, top floor to bottom floor


naming in the rulesets would not change:
    NAME - old .MAP/.RMP stuff.
    THEMAPNAME - new-style maps, override the old-style ones,
                     and they don't pay any attention to what missiondef
                     says about terrain. Implicit terrain stuff is only
                     for the old-style maps.



It would allow text editing of routes only, the rest would be read-only, but still informative.
(can't change terrain sequence without changing the binary data or hilarity ensues)

maybeExpectedTerrainSetSizes can record MCD set sizes at creation, so changing that after the map was baked could be caught.


Offline ohartenstein23

  • Commander
  • *****
  • Posts: 1931
  • Flamethrowers fry cyberdisk circuits
    • View Profile
Re: MAPVIEW upgrade
« Reply #369 on: February 09, 2019, 07:09:09 pm »
We already have a way around this in OXCE - you can combine multiple terrains in a single map script (terrain tag in a mapScript command), or layer multiple maps with different terrains (verticalLevels tag in a mapScript command).

Offline Stoddard

  • Colonel
  • ****
  • Posts: 485
  • in a fey mood
    • View Profile
    • Linux builds & stuff
Re: MAPVIEW upgrade
« Reply #370 on: February 09, 2019, 08:20:29 pm »
We already have a way around this in OXCE - you can combine multiple terrains in a single map script (terrain tag in a mapScript command), or layer multiple maps with different terrains (verticalLevels tag in a mapScript command).


Um, well, I overlooked that development.

Still I think it'd be better to keep terrain definition closer to the map.

Offline kevL

  • Colonel
  • ****
  • Posts: 466
  • pitchforks and torches
    • View Profile
Re: MAPVIEW upgrade
« Reply #371 on: February 09, 2019, 08:52:05 pm »
I am wondering if i am getting into a issue with the following MCD file group. Since the asigned PCK's are over 256.

a Mapfile doesn't store Pck entries, it stores Mcd record ids (which then have pointers to Pck data)

so the quantity of images isn't limited by the Mapfile format;


Still I think it'd be better to keep terrain definition closer to the map.

you're probly right -- though that's a decision for OXC/E
« Last Edit: February 09, 2019, 08:58:38 pm by kevL »

Offline hellrazor

  • Commander
  • *****
  • Posts: 2013
  • Deep Ruleset Digger & Bughunter
    • View Profile
    • Github Account
Re: MAPVIEW upgrade
« Reply #372 on: February 10, 2019, 03:34:04 pm »
a Mapfile doesn't store Pck entries, it stores Mcd record ids (which then have pointers to Pck data)

so the quantity of images isn't limited by the Mapfile format;

Good to know then i can make this UFO =)

Offline tkzv

  • Commander
  • *****
  • Posts: 583
    • View Profile
Re: MAPVIEW upgrade
« Reply #373 on: February 11, 2019, 12:53:14 am »
Thanks for trying to make this work in Mono, kevL.

To answer your question on GitHub: I treated all black pixels as transparent, because not drawing only colour #0 did not have the desired effect. As far as I can tell, you replaced my check with (palid != Palette.TransparentId) in case the transparent colour isn't 0. This makes sense. Unfortunately, this doesn't work for me. I downloaded your version ab4868e, compiled it and again got black rectangles.

UPDATE: My bad, forgot to define LOCKBITS. Now it looks more or less fine. Does it have to draw gaps between tiles? Also, when I move mouse over the window, a number of horizontal lines appear, vaguely in a shape of the craft displayed. What may be causing them?

UPDATE 2: Found "Use Mono" option. Now everything is fine.

Stoddard's latest build https://lxnt.wtf/oxem/builds//MapView/MapView-ab4868ec-2019-02-08-mono.7z does not work in my system (Ubuntu 16.04), giving the error:

For System.MissingMethodException: Method 'String.Format' not found.
  at YamlDotNet.RepresentationModel.YamlStream.Load (IParser parser) <0x40cbb760 + 0x0003b> in <filename unknown>:0
  at YamlDotNet.RepresentationModel.YamlStream.Load (System.IO.TextReader input) <0x40cb99b0 + 0x00047> in <filename unknown>:0
  at MapView.XCMainWindow.LoadOptions () <0x40cb7f20 + 0x0013b> in <filename unknown>:0
  at MapView.XCMainWindow..ctor () <0x40c17b80 + 0x006bb> in <filename unknown>:0
  at (wrapper remoting-invoke-with-check) MapView.XCMainWindow:.ctor ()
  at MapView.Startup.RunProgram () <0x40bd6fe0 + 0x000db> in <filename unknown>:0
« Last Edit: February 11, 2019, 01:10:56 am by tkzv »

Offline kevL

  • Colonel
  • ****
  • Posts: 466
  • pitchforks and torches
    • View Profile
Re: MAPVIEW upgrade
« Reply #374 on: February 11, 2019, 03:43:06 am »
Thanks for trying to make this work in Mono, kevL.
np. you did the groundwork, i had the time, it was interesting ....... tks.

Quote
To answer your question on GitHub: I treated all black pixels as transparent, because not drawing only colour #0 did not have the desired effect. As far as I can tell, you replaced my check with (palid != Palette.TransparentId) in case the transparent colour isn't 0. This makes sense.

note that (const)Palette.TransparentId is defined as "0" in the code elsewhere -- it's not, eg, getting the transparency ID out of an image and using that, it's just avoiding the use of "0" (as a magic number).

The thing is, if you want to test against palette ids, this is the necessary step:

Code: [Select]
    palid = bindata[++i];

bindata is retrieved from an XCImage object, in this case a terrain-sprite at the right animation frame

ps. LOCKBITS is an experimental routine that draws sprites pixel by pixel; but i didn't notice any speed increase so its on hold indefinitely.


Quote
UPDATE 2: Found "Use Mono" option. Now everything is fine.
i guess I should make note of that on the Distribution page ... a bit of my bad


Quote
Stoddard's latest build https://lxnt.wtf/oxem/builds//MapView/MapView-ab4868ec-2019-02-08-mono.7z does not work in my system (Ubuntu 16.04), giving the error:

For System.MissingMethodException: Method 'String.Format' not found.
  at YamlDotNet.RepresentationModel.YamlStream.Load (IParser parser) <0x40cbb760 + 0x0003b> in <filename unknown>:0
  at YamlDotNet.RepresentationModel.YamlStream.Load (System.IO.TextReader input) <0x40cb99b0 + 0x00047> in <filename unknown>:0
  at MapView.XCMainWindow.LoadOptions () <0x40cb7f20 + 0x0013b> in <filename unknown>:0
  at MapView.XCMainWindow..ctor () <0x40c17b80 + 0x006bb> in <filename unknown>:0
  at (wrapper remoting-invoke-with-check) MapView.XCMainWindow:.ctor ()
  at MapView.Startup.RunProgram () <0x40bd6fe0 + 0x000db> in <filename unknown>:0

hm, appears Mono doesn't interpret something in YamlDotNet quite right. perhaps @Stoddard can give it a run and get the line # of the offending String.Format() ? Does it happen with an empty /settings folder? Ie, try deleting MapViewers.yml (window positions)



hm, i googled for "mono MissingMethodException string.format"

a couple of possible leads ...

a) Unable to execute build.sh in Ubuntu due to "Error: Method 'String.Format' not found."
https://github.com/cake-build/cake/issues/1929
(resolution: upgrade Mono)

b) Method not found: 'System.String System.String.Format(System.IFormatProvider, System.String, System.Object)
https://stackoverflow.com/questions/30558827/method-not-found-system-string-system-string-formatsystem-iformatprovider-sy
(resolution: force a different overload for String.Format())


here's a better stacktrace of what i think is going on,
https://github.com/kevL/OpenXCOM.Tools/blob/master/YamlDotNet/Core/ParserExtensions.cs#L46
called from
https://github.com/kevL/OpenXCOM.Tools/blob/master/YamlDotNet/RepresentationModel/YamlStream.cs#L100
called from
https://github.com/kevL/OpenXCOM.Tools/blob/master/MapView/Forms/MainWindow/XCMainWindow.cs#L573

which means parser.Allow<T>() is null, which means parser.Accept<T>() is false, which means parser.Current is *not* T, which throws a YamlException with the string.Format() call that borks.

Try deleting MapViewers.yml
« Last Edit: February 11, 2019, 03:50:25 am by kevL »