Hello,
In this video I try and answer all of your questions and I also simply try and explain what it is that I would like to try to add/implement into OpenXCom:
Lot's of stuff is covered in this video, but it is well worth a watch, at least I hope so, and that you can understand me somewhat:
https://www.youtube.com/watch?v=ljwRXdXmhPkFeedback is welcome, further ideas, pointers, tips, links, new questions, algorithm questions/improvements, remarks, insight, anything is welcome ! =D
Planning, Goals achieved, Time/Day spent:
* Last Update 20 may 2022 *:
1. fast voxel traversal algorithm: unlimited, must get it workingstarted: Tuesday 8 march 2022
finished: Thursday 24 march 2022 (in delphi and same day c/c++)
debugged it in c/c++: Friday 25 march 2022, days spent so far: 17 days.
2. line box intersection: 5 days started: Tuesday 22 march 2022
mostly finished: Friday 25 march 2022, days spent so far: 3 days but from those 17 days above.
Saturday 26 march 2022: acquired new cable modem in city.
Sunday 27 march 2022: installed new cable modem at home.
Monday 28 march 2022: slept ?
Tuesday 29 march 2022: refactored fast voxel traversal algorithm, cleaned up code, tried to seperate code into process voxel method but failed and reverted it back. + 1 day spent.
Tuesday 29 march 2022: started work on new shading engine, implemented GenericMap template and basis/constructor/destructors and maps for ShadingEngine class
*overlap*:
Tuesday 26 april 2022: started work on a new ray segment box intersection algorithm, which will be integrated into (grid) traverse algorithm/data structures for efficiency and correctness sake.
Saturday 30 april 2022: almost perfected new ray segment box intersection algorithm, with floating point exceptions off it's now perfect.
Sunday 1 may 2022: explained the ray segment vs box intersection algorithm which needs something extra vs just ray vs box intersection algorithm. Created my own theoretical version to prove the algorithm, the theory, it works.
Tuesday 10 may 2022: (5 hours and 15 mins) Documented/described the (new) line/ray segment multi-grid traversal algorithm. Multi-level distance compatible traversing also using distance parameters from the new ray segment box intersection algorithm (both to maximize efficiency/re-use calculated variables):
3. grid space partioning: 7 days4. growing bounding boxes: 7 days5. cut voxel data with sprites: 14 days6. light/shadow casting: 21 daysSaturday 2 april 2022: experimented with painter algorithm and acquiring surfaces for shading engine, not too usefull (yet? maybe in near-future though)
Sunday 3 april 2022: experimented with trying to collect data from painther algorithm via ShaderDraw method, was too complex.
Monday 4 april 2022: investigated draw terrain method. (Replaced/Reinvented blittNshade/shaderDraw method with simple draw sprite method)
Tuesday 5 april 2022: made it possible to draw 3d voxels/grid inside a sprite for draw terrain method (see DrawVoxelDataInsteadOfSprite)
Thursday 7 april 2022: good attempt at drawing lofts/voxel 3d + first attempt at light casting/shadow casting which was very slow and perhaps not so good yet, might need debugging. (In shadow/black yes/no for now, no distance or angle shading yet)
Friday 8 april 2022: Tried to implement multi-level/dual-level fast grid traversal. Attempt 2 was somewhat successfull, but now flickering/blinking of border cells is back, and also catched some division by zero corner case. I am considering to implement it differently using existing code to make sure everything works they way it should, by make code re-useable and then using it twice or so.
Saturday 9 april 2022: Investigated possibility of using Physically Based Rendering Technology engine version 3 and version 4, version 3 works but is slow to render, it could be used to render a cool single scene/picture of an xcom scene.
Sunday 10 april 2022: Build some new vcl engine to test realtime graphics in preparation for perhaps grid movement and control points on grid and line, then had too much fun implementing transformer graphics to test it and spent too much time trying to get audio stuck/asio 1.3 and 1.4 working for blaster vst but it was fun.
Saterday 11 paril 2022: Implemented SpriteVoxelFrame which accelerate the finding of a voxel behind a sprite/tile pixel. (Took 15 hours to implement: study code, create code, debug code, test code, I made an interesting discovery, apperently youtube/obs studio streaming limited to 12 hours, last 3 hours were cut off, but fortunately success was recorded !
)
(I was somewhat frustrated the last 2 or 3 days with the multi-level traversal algorithm but feel ok now. Decided to go straight into C/C++ and do some usefull work there. The plan is to re-use the C/C++ code to try and re-implement multi level traversal algorithm with existing code and ripping it apart twice for usage on openxcom scenes for tracing of lines/rays. Idea is to trace the rays in parallel step by step for maximum data access re-usage/locality. This is a more efficient usage of my time to make progress in trying to implement this feature as far as possible.)
I am happy and statified with the progress I made on 11 april 2022.
Wednesday 13 april 2022: implemented ScreenVoxelFrame (might need masks to not shade things which don't have voxel maps or units on top of it, for now I go with the flow
)
started work on tile + voxel traversal code.
Friday 15 april 2022: VoxelRay.h and VoxelRay.cpp header/implementation continuation work: approx 4 hours. Later on the day 2 hours and 30 mins: added ScreenRayBlocks and first usage code and light exposure + angle + shade code/sketch/idea.
saturday 16 april 2022: rested a lot, also Described a new algorithm idea for processing/traversing multiple light sources efficiently by avoiding unnecessary light rays.
Also described initial idea for a more advanced voxel engine in the future. Also sun, earth, moon, dark/full moon idea and such, see video links down in postings below.
Sunday 17 april 2022: Debugged VoxelRay.h and VoxelRay.cpp for tile traversing, this should now work very well, spent 4 hours and 50 mins on it.
Monday 18 april 2022: Spent 14 hours trying to debug the "voxel traversal", which comes after the "tile traversal". Also tried to do a re-design of the code, experimented with c/c++ language features/templates. So far a mysterious bug remains.
Tuesday 19 april 2022: Rendering bug with voxel 3d map solved (called in wrong place). Tile/Voxel Traversal seems to be working good and should be in a usuable state for light casting/shadow casting/shading, but should still be tested a little bit more to be absolutely sure. Grid to ASCII tool created. Described the Tile and Map axis and coordinates on a forum posting and file.
Wednesday 20 april 2022: Spent 10 hours trying out light casting attempt 2 and 3 and also directional tile lighting, not much success so far.
Thought light casting attempt 2 shows some promise, light rays visible in trace, however there is something wrong with the screenvoxelframe or
something with the sprite frames, something weird going on, as if lower air tiles have walls or something.
Friday 22 april 2022: Spent 10 hours. Some significant progress has been made. For the first time something that resembles "perspective" shadows can be seen. Hold control + L to see it in action. Tested voxelray traversal for full x, y and z lines, seems to be working ok. Could be an interesting technique maybe for "isometric" shadows.
Sunday 24 april 2022: (Spent 5 hours) The screen voxel frame idea is finally working. Though some problems remain. There are gaps/lines between the sprites. Most likely the double vertical line in sprites in the center.. A possible solution could be to try and replicate this line inside the voxel 3d maps themselfes.
Tuesday 26 april 2022: Worked 8 hours on new ray segment box intersection test. (Working 99.9999% correct I think it is useable for openxcom !
) (Also tried out strong type checking for different dimensions x,y,z and used delphi's record operator overloading, cool stuff. Spent 2 hours or so on that.) I also learned how to turn off annoying floating point exceptions in Delphi.
(Took some time off to recover from depression from my shitty graphics experiments lol... (the graphics experiments were worth it though, "it's all part of the game/improvement program ! =D" played some civ1 lol... hopefully the openxcom/light casting graphics will get better over time as I learrrrnnn and improve code and methods and algos !
=D)
Saturday 30 april 2022: (16 hours spent) Almost perfected the ray segment box intersection algorithm. (In principle it now runs flawlessly if exception mask is set to raise no floating point errors. (integrated it also with traverse fields) The ray segment box intersection algorithm can now handle a single ray point, a ray on the left,top,right,bottom boundary and ofcourse inside too and intersecting ofcourse. So small extra work could still be done to make it run flawlessly if floating point exceptions are enabled by checking direction.x,y,z for zeros and writing custom/solution code for this situation, I may/could still look into this further.
Sunday 1 may 2022: (Spent 5 hours or so): Explained the difference between the ray and ray segment vs box intersection algorithm. Created and perfected my own theoretical version to prove that I understand the algorithm once and for all, 100% and thank god I did do that ! =D
Intermezzo 1:
Then watched "The Batman " ! possibly one of the best movies I have ever seen. The ambient/mood/visuals/vibe are stunning ! Definetly a must watch for everybody ! =D
(Now today Monday 2 may 2022... I am probably going to watch Picard season 2 for a long time ! =D I have some catching up to do !
=D)
Further plan for Monday 2 and Tuesday 3 may 2022:
And then if time, energy and clearity/motivation allows it I will probably work some more on the ray segment box intersection algorithm so that it can run with floating
point exceptions enabled, so that means taking care of all the divide by zero problems.
Tuesday 3 may 2022 took some time to recover from watching 9 episodes of Picard lol.
Wednesday 4 may 2022 probably took some more time off.
Thursday 5 may 2022 spent 8 hours analyzing difference between khabarovsk and zorkiy ship in world of warships.
My motivation to return back to this OpenXcom light casting algorithm is dwindling because I am starting to think/believe it might not look as good as I had hoped,
though I have not yet giving the shading formula a good chance. Plus I may have already spent the planned 50+ minimum ammount of days, plus I have to go do
some other things the coming days. So at least the next few days I will not be doing any coding and taking more time off to get some life things done !
Also still need to see the last episode of picard season 2 ! =D
Plus I am kinda longing back into world of warships ! for more multiplayer challenges, the AI in Civ1 is starting to bore me a little bit and the AI in OpenXcom is
probably not that interesting to play against.
Friday 6 may 2022: Spent 41 minutes trying out a potential fix, but it was not the core of the issue. I am open to any suggestions what might be the problem !
Here is the video:
https://www.youtube.com/watch?v=nkoHfNEbY24(Now I have to go do other things, later !
Saturday 7 may 2022: (8 hours spent) First impression of what it could look like, moving sun added, custom shading for lit and unlit, double floor voxels removed from tile.voxelmap3d. Some floor/traversal issues seem to remain for now.
Intermezzo 2:
Sunday 8 may 2022: Went to the super market
Monday 9 may 2022: Visitted my mother
Tuesday 10 may 2022: (5 hours and 15 mins) Documented/described the (new) line/ray segment multi-grid traversal algorithm. Multi-level distance compatible traversing also using distance parameters from the new ray segment box intersection algorithm (both to maximize efficiency/re-use calculated variables):
https://www.youtube.com/watch?v=Te0nmf70WN8Thursday 12 may 2022: (4 hours spent) 1. Fixed three floating point problems with new ray segment box intersection algorithm. 2. Started work on implementing isometric mode which may have to diagnose 3D problems and also allows testing everything in 3D ! 3. Played a little bit of OpenXCom to see how tiny OpenXCom truely is on todays monitors ! =D So far so good, very cool stuff ! =D
https://www.youtube.com/watch?v=_YnWLl-3Nac Saturday 14 may 2022: (Spent 7 hours 15 mins) Wrote a special application in Delphi to analyze/inspect OpenXcom palette and extend/grow/scale it to a more advanced palette (so that each row has more shades available) with linear and cubic interpolation. The cubic interpolation needs a little bit more work, for begin and end sections, the end section might also be slighty buggy maybe ?!
https://www.youtube.com/watch?v=myqxefLcoOoSunday 15 may 2022: Tried a 24 bit version of OpenXcom. (Width is wrong) Changed OpenXcom palette analyzer to point series to illustrate the difference in palette quality.
Monday 16 may 2022: Tried yankers 32 bit support, investigated the possibilities of PBRT. Achieved/build an operational version of PBRT v4 win32. (Yaml is missing/link errors)
Tuesday 17 may 2022: (8 hours and 45 mins): Investigated the possibility of using PBRT as a static library:
Wednesday 18 may 2022: Spent the whole day messing around with trying a static library version of PBRT, failed with link errors
Also spent time learning the PBRT file format.
Thursday 19 may 2022: (8 hours and 30 mins spent) Investigated PBRT rendering of OpenXcom map and isometric view, also write an OpenXcom exporter to PBRT format, which exports the game in real time (first draft/working version). First render of OpenXcom in PBRT ?!
Friday to Saturday Night 20 to 21 may 2022: (Spent 3 hours) Cleaning up code and re-writing PBRTExport to try and generate PBRT files more efficiently, however the code has become very slow, probably because of CPU L1 cache trashing, this problem will have to be solved tomorrow/next time.
(MAYBE I AM WASTING MY TIME ON PBRT, BUT I AM GOING TO TRY IT ANYWAY, BECAUSE IT IS INTERESTING)
Yes PBRT was a big waste of time, it's a very messy project and link problems, The same can be said for other tracers, like mitsuba 2, problems with python, and intel otp probably takes forever to build and probably won't run in Win32 builds. So I am going to quite with noisy images and shitty to build and link raytracers and back to my own code !
7. light reflection/radiosity: 21 days8. physics: 7 days.
9. export game to big voxel map: 10 days
10. loading custom voxel graphics (also for colissions): 38 days
11. dithering: 7 days
12. Transparency: 7 days.
13. Rain: 5 days.
-------------------------------------------------------------------
Minimum goals that I want to achieve (do this or die trying =D):
2. line box intersection: 5 days, achieved.
3. grid space partioning: 7 days
4. growing bounding boxes: 7 days
5. cut voxel data with sprites: 14 days
6. light/shadow casting: 21 days
-------------------------------------------------------------------
Minimum days: 54
-------------------------------------------------------------------
Interesting extra goal:
7. light reflection/radiosity: 21 days
Easy extra goal:
8. physics: 7 days.
-------------------------------------------------------------------
Interesting+Easy days: 28 days.
-------------------------------------------------------------------
Extra/Optional goals:
9. export game to big voxel map: 10 days
10. loading custom voxel graphics (also for colissions): 38 days
11. dithering: 7 days
12. Transparency: 7 days.
13. Rain: 5 days.
--------------------------------------------------------------------
Extra days: 67
--------------------------------------------------------------------
Total days: Minimum Days(54) + Int/Easy(28) + Extra/Optional (67) = 149
(MAYBE YOU CAN HELP ME TO DECIDE WHERE TO PUT ANY NEW CODE, IN WHAT FILES ?)
(ADDITIONAL MUST HAVE FEATURE: ROTATING THE MAP TO LOOK AT IT FROM DIFFERENT
TACTIC PERSPECTIVES/ANGLES)
NEW/ADDITIONAL GOAL:
14. INVESTIGATE VOXEL COLLISION DETECTION, DETERMINE ACCURACY, PERHAPS
REPLACE AS WELL WITH FAST VOXEL TRAVERSAL ALGORITH !!!
15. Explore the possibilities of using an existing "ray tracer", especially the physically based rendering one seems interesting, especially for the hopefully easier light/multiple light implementation/support.
Bye for now,
Skybuck.