Feature: Revamped research progress screen (QoL)

[krakp]

// Not sure if this belongs here or to the Programming part - feel free to move if necessary

Would the community be interested in research screen showing the man days left per project as a range?



This is purely a QoL feature - it does not reveal the actual roll nor any other info that you wouldn't be able to find out without cheating. It basically uses the average completion time, whether the progress is known or not and the already invested man days - all of these you can track yourself with a pencil or an excel sheet. This feature does it for you.

I already implemented it for myself and happy to share if there is interest (and if you tell me "how" I should integrate it).

[0xEBJC]

Would the community be interested in research screen showing the man days left per project as a range?

...
...

I already implemented it for myself and happy to share if there is interest (and if you tell me "how" I should integrate it).

Nice, I thought of adding something like this as a percentage xx% done.  Although that idea or your implementation is very helpful, to me it feels more authentic and realistic for "researching" without knowing how long precisely to finish.  That's why I ended up not perusing this.  you could always have this as a toggleable feature in the advanced options?

[Finnik]

I personally don't like when there is a straight progress for research shown to a player. I feel it is nice to have it separate from manufacture, where you can plan things. Research progress is very hard to plan, as there is a lot based on insight. I have not been personally involved into a "big" science projects IRL, but I am familiar how it is done in details - it is very hard to plan in terms of end date.

So I am against it.

[krakp]

Just to make sure that everybody is on the same page on this. This feature does not give you any more 'accuracy' than the standard xcom display - it just tracks the already invested time into the project and discounts it from the possible range. As you can see in the screenshot, the ranges for the Motion Scanner and MediKit are still huge - so other than seeing them in a digital form, there is not really much difference.

For the Laser Weapons I got a 'lucky roll' - the progress was known at the very start. Therefore the range is way tighter.

[Meridian]

Not everybody has studied the guts of xcom and knows that "Unknown" for motion scanner means 121-268 days with two scientists allocated and X persondays spent already.
"Unknown" literally means unknown, i.e. the player should not have any idea if it is 10 days or a million days.

The original game OBVIOUSLY didn't want to show any details other than the T-shirt size estimate (S, M, L, XL).

I also think that it's much better when it says unknown, poor, average, good, excellent... instead of magic numbers; or any numbers for that matter.
IMO, the proposed feature is an anti-feature.

PS: if someone wants to study the guts, and minmax with pen and paper, or cheating, or with anti-features... we're not stopping you, but calling this a FEATURE (in capital letters) or QoL is... inappropriate.

[krakp]

calling this a FEATURE (in capital letters) or QoL is... inappropriate.

The CAPS already removed - no idea why I posted it this way - feedback taken :-).

The original game OBVIOUSLY didn't want to show any details other than the T-shirt size estimate (S, M, L, XL).

I fully agree - I also once enjoyed the 'romantism' of not knowing the research tree and/or the average research times. And I also agree that for folks fresh to XCOM or fresh to any other mod (with a different research tree) they may not want to ever see anything beyond the high level estimates.

However, for the seasoned XCOM players (and I guess there are many of them here), the knowledge from https://www.ufopaedia.org/index.php/Research_Technical_Details is just part of their gameplay (the same way as any strategies, weapon usage, base layouts etc). So while it indeed comes from the guts of XCOM, today these are the "known guts" - additionally the Tech Tree Viewer feature (not sure if it was introduced in OXC, OEXCE or BOXCE - you get it by clicking the middle button of the mouse on the research screen for a Tech) also already shows the average completion time for each Tech (using a bit encoded combination of "#, =, - " chars - but the info is there, I assume also for custom mods...).

So what my feature does, is just decoding the info that is already present in the game - that's the reason why I called it a QoL.  It does not cheat, it does not have access to your actual roll, it does not do anything inappropriate, in my opinion. It just simplifies the tracking of the research progress, the same way as other (great features) simplify the tracking of the radar range, TU range, soldiers stats or equipment weight vs strength.

And again - I already implemented it for myself (and I was only able to do this thanks to all the great work you guys did on (B)OXC(E) - chapeau to all of you), so the ask was only whether other people would also see this as a feature :-).

[Meridian]

it does not have access to your actual roll, it does not do anything inappropriate, in my opinion.

You said something different just a few posts above:

For the Laser Weapons I got a 'lucky roll' - the progress was known at the very start. Therefore the range is way tighter.

Sounds like it has access to your actual roll, and reveals it.

[krakp]

Sounds like it has access to your actual roll, and reveals it.

Not quite. In the original XC (refer to https://www.ufopaedia.org/index.php/Research_Technical_Details) the known/unknown indicator would change depending on the "effort still needed" - whenever it gets to less than 66% of the average needed effort you get the known status. This would have the interesting side effect that you would sometimes (16% of the time) get such a lucky roll that you would get the known status at the very start (the "Known At" value from the ufopaedia.org article). Of course knowing that your status is already known means that the initial roll must have been between the Min and KnownAt - which is a much smaller range than the one from KnownAt to Max. So you could indeed argument that the algorithm needs to know the roll to switch from unknown to known, but that is the same as in original XC.

I know that the current OXC implementation is different (it shows the known status once you invest more than 33% of the average effort), but my approach was to mimic more the original algorithm. In the original XC the switch from unknown to known would tell you that you approx. still need to invest  2/3 of the average (which is quite a good indication for progress, from  which you can draw conclusions and reallocate scientists, if you want). In the current OXC version, the fact that it switches from unknown to known means basically nothing - since you can track how much time you invested and can check the average project completion using the middle button, you can exactly tell when this switch will happen. It is therefore not very informative...

[krakp]

The logic inside original XC is on a high level described here:

https://openxcom.org/forum/index.php/topic,10910.msg161686.html#msg161686

[Meridian]

You don't need to quote original XCOM to OpenXcom devs, we are all 100% vanilla xcom fanatics here.

I've played the game a million times and I've studied the guts more than I should have too.

The logic in OpenXcom is based on the reversed code from the Windows Collector's Edition (not the DOS edition).

Here's the code:

Code: [Select]

      if ( p_GS_Project_DAT[v_Base_LocIdx].ProjectScientists[v_Dialog_Processed_Item_Idx__] )
      {
        v7 = a_GS_Research_Builtin[v_Dialog_Processed_Item_Idx__].time_to_research;
        v2 = (unsigned int)(0x55555556ui64 * a_GS_Research_Builtin[v_Dialog_Processed_Item_Idx__].time_to_research >> 32) >> 31;
        v3 = v7 / 3;
        if ( v7 - p_GS_Project_DAT->ProjectDays[v_Dialog_Processed_Item_Idx__ + 144 * v_Base_LocIdx] <= v7 / 3 )
        {
          v38 = 1;
        }
        else
        {
          v8 = p_GS_Project_DAT[v_Base_LocIdx].ProjectScientists[v_Dialog_Processed_Item_Idx__];
          v2 = 100 * v8 / v7;
          v3 = 100 * v8 % v7;
          if ( (_WORD)v2 <= 25 )
          {
            if ( (_WORD)v2 <= 13 )
              v38 = ((_WORD)v2 > 7) + 2;
            else
              v38 = 4;
          }
          else
          {
            v38 = 5;
          }
        }
      }
      else
      {
        v38 = 0;
      }

If you're wondering about the constants assigned to variable v38, they are as follows:

0 = None
1 = Unknown
2 = Poor
3 = Average
4 = Good
5 = Excellent

Edit: tried to improve wording

[krakp]

we are all 100% vanilla xcom fanatics here.

Double thumbs up here!!! (not finding the right emo in the forum ). You've all done some crazy magic here that enabled me (and so many other folks) to not only come back to a so much better version of my favorite game, but also to think about ways how I could make it even more interesting. Not only to think - to actually implement these changes. You are all my HEROES!
I hope I will be able to pay it back to the community one day - I am fully aware that until now I am just asking questions all around. The OXC universe is quite big and challenging at the beginning - but slowly I am learning my routes. Maybe soon I will be able to answer some questions on the forum or even help implement some feature requests? 

Coming back to the research discussion - I never played the CE version (I am on the GOG version of X-Com - I guess this is the DOS one). What I can tell you though (and I actually tested it again very recently on my vanilla game) is that indeed if you test starting the 3 initially available projects enough times, you will get one of them to immediately show Poor/Good/Average, depending on assigned scientists (so basically immediately jump into the Known state). Maybe you can test it on the CE version too (or whatever version you own) - if you restart the game a few times, you should for sure get at least one immediately known project (the chances are 1 to 6). That's exactly what https://www.ufopaedia.org/index.php/Research_Technical_Details describes. And this can never happen in the current OXC implementation - cause it checks for SpentDays - which are always 0 on start.


As for the code you shared: (I removed the 'dead' lines calculating the v2 and v3, that are never used and added the comments to show which part of the IF represents KNOWN)

Code: [Select]

      if ( p_GS_Project_DAT[v_Base_LocIdx].ProjectScientists[v_Dialog_Processed_Item_Idx__] )
      {
        v7 = a_GS_Research_Builtin[v_Dialog_Processed_Item_Idx__].time_to_research;
        if ( v7 - p_GS_Project_DAT->ProjectDays[v_Dialog_Processed_Item_Idx__ + 144 * v_Base_LocIdx] <= v7 / 3 )
        {
          v38 = 1;  // UNKNOWN
        }
        else
        { // KNOWN
...

v7 seems to be the Average Days for the project.

p_GS_Project_DAT->ProjectDays[v_Dialog_Processed_Item_Idx__ + 144 * v_Base_LocIdx] is probably the days left for this particular project execution (which implicitly contains the current roll - only by knowing it you can figure out the time left).

So the condition under if translates to:

Average_days - Days_left <= Average_days / 3

Thus:

Average_days *2/3 <= Days_left

And this is exactly the algorithm I described (that I learned from ufopaedia.org). If the days still left (the effort needed) is more than 66% of of the Average_days the game shows Unknown. As soon as you progress so that you have less than 66% of the Average_days left, the progress becomes known.

Please go through this analysis in peace and correct me if I am wrong - however, it does seem to add up quite nicely and to be aligned with the info from ufopaedia.org.


Just to make sure that I do not come across as Mr. SmartyPants here - the current OXC implementation is also very logical and I can fully understand that many people would like it. It is just that for my use case (where I try to optimize my research time using the info that the game gives me) it felt like a huge handicap compared to vanilla. So I went ahead and changed the implementation (for myself). But while analyzing the algorithm (to implement it properly) I realized that using the info the game gives you, you can actually figure out much more than just Poor or Excellent. So I first started calculating it manually (with the proverbial pencil) and then spent some hours to actually implement it.

And again I am happy to share the code if there should be enough interest. Hey, I could even offer 2 options - getting the Unknown, Poor, Excellent, but according to the algorithm from XC or getting the number ranges from the screenshot. Maybe this would be a way for me to pay back at least a part of my debt to this community :-). Cheers!

[Meridian]

Sigh.
You're missing my point.

I'm not questioning your testing.
All this has been tested by many people (including you and me), and confirmed and re-confirmed countless times.
You are factually correct, nobody questions that.

But, take a step back, and look at that (reversed) code from afar, how it is structured, what is calculated there... why would they calculate "v7 / 3"... twice... if they didn't want to use that value for comparison?

We don't have the actual code unfortunately (nor any code comments), so we don't know how well the higher-level structure matches... but I am positive it matches sufficiently and that the design intention was something similar or identical to what Openxcom has done. And the difference... is a bug.

As I already said earlier, feel free to take advantage of this (but don't expect this to be implemented in OXC/E).

[krakp]

I was indeed not getting your point  - I never participated in the previous discussions on this topic. But I think i am getting it now - what you are saying is that the whole "KnownAt" theory (that I was referring to and that is described in ufopaedia.org) is a result of a bug in the implementation of XC. And that the current implementation in OXC fixes this bug. Therefore the page https://www.ufopaedia.org/index.php/Research_Technical_Details just describes "exploits" of this bug and my approach is basically doing the same.

Not sure how much I (want to) agree with this :-). I actually like this original "buggy" design and I am not a huge fan of the way OXC did it. That's why I put the effort to rewrite it...

Some comments (that will probably not convince you, but hey, let me at least explain where I am coming from):
- this was never mentioned as a bug before OXC: https://www.ufopaedia.org/index.php/Known_Bugs
- nor is it listed as a deliberate difference from XC https://www.ufopaedia.org/index.php/Differences_to_X-COM_(OpenXcom)
- the initial check-in https://github.com/OpenXcom/OpenXcom/commit/3e09834ede1590bcf932363ea48fd9b1a9af3a12 contains no comments, so we can't really know how much intentional this change was (or did the author actually clarify this on the forum?)
- the code has never been touched since the initial check in (13 years ago)

So my gut feeling is that the initial OXC design (that maybe actually tried to mimic the original XC design, but did not quite make it), was just deemed good enough (which it definitely is - all in all, the code is stable for so long....).

My main challenge with this design is the following:
- the switch to the 'known' just tells you the amount of man days that you already invested, which you can know anyway if you remember when you assigned how many scientists. So it kind of feels useless. In original XC the switch to 'known' is extremely useful - it tells you that you maximally have 2/3 of the average time left.

And since the other labels after the progress is known are equally useless (but this both in XC and OXC) - they only show you how the amount of assigned scientists relates to the average_days (which you can easily calculate yourself), I thought that having "honest" ranges would get rid of all this weird ambiguity.

Anyway - I will keep this code private and probably look for other opportunities to contribute in the future.

Cheers! :-)

[Meridian]

Yes, exploits are exploits, and there is no argument that validates an exploit for me.

The more "unknown" research approach that this currently implemented in OXC or rather the more "known" one from XC (in this case the ability to cancel and restart projects should be removed - otherwise one could retry until you get the progress and know that you got a good roll - btw. why would projects be "cancellable" anyway? I never missed this possibility in XC....).

You are aware of the exploit, so I won't try to convince you, but I'll paste this also here in this thread for others.

Btw. removing the Cancel ability doesn't solve anything.
You can still start the research project in 8 different bases (even in vanilla you can have 3 lab bases (with 1 scientist) by end of January, and 8 lab bases by end of March).
This gives you 8 rolls for each research, making it in average more likely than not to get a lucky roll with known progress in at least one of them (then just transfer the rest of the scientists and finish the research in the most favorable base).
Basically you could cheat all your mid-game and late-game research.

In mods, the situation would be even worse.
You get 8 bases comparatively a lot sooner than in vanilla, so basically all your research including early-game is exploitable.

[krakp]

The "multiple bases" argument is very convincing :-). It did not occur to me as something very relevant, since I rarely have more than 1 base, however, indeed for mods with many bases it is a very valid concern. Interestingly I noticed that the Ufopaedia article I referred to also mentions this exact scenario:
"the only way to exploit Knowns "honestly" would be to try the project at additional bases. However, it would be a hassle to set up labs at lots of bases, especially given the fact that it only takes 3+ Labs a few months to do most of the research (see Total Time, below)."
In vanilla game it indeed rarely makes sense - for mods when you get bases sooner or the research takes way longer, this is a very valid concern. So I agree that it may be perceived (and used) as an exploit. Still, if someone would be interested in getting the code "in private" - just PM me.


Now that this topic is clarified, I would maybe have another (hopefully much less controversial) suggestion :-). While working on my research feature I noticed how the initial roll for a project is calculated in https://github.com/Xilmi/OpenXcom/blob/oxce-plus/src/Basescape/ResearchInfoState.cpp

Code: [Select]

        int rng = RNG::generate(50, 150);
int randomizedCost = rule->getCost() * rng / 100;
if (rule->getCost() > 0)
{
randomizedCost = std::max(1, randomizedCost);
}
_project = new ResearchProject(rule, randomizedCost);

Since we are only choosing from a 100 (or 101 to be exact) random values, the granularity of the roll will be a 100th of the Average cost. So e.g. for the Blaster Launcher project (average 900) you can get a roll of 900 or 909 but you will never get anything in between.

I improved this granularity (so that you can get every integer value from the possible range) by changing this function to a much simpler version:

Code: [Select]

       int randomizedCost = RNG::generate(rule->getMinCost(), rule->getMaxCost());
       _project = new ResearchProject(rule, randomizedCost);

For this to work I introduced 2 very simple auxiliary functions in src\Mod\RuleResearch.cpp:

Code: [Select]


/**
 * Gets the minimum cost of this ResearchProject.
 * @return The minimum cost of this ResearchProject (in man/day).
 */
int RuleResearch::getMinCost() const
{
return (getCost() + 1) / 2; /// for integers -  (x+1)/2 is the same as ceiling(x/2).
}

/**
 * Gets the maximum cost of this ResearchProject.
 * @return The maximum cost of this ResearchProject (in man/day).
 */
int RuleResearch::getMaxCost() const
{
return ((getCost()) * 3 + 1) / 2; /// for integers -  (x*3+1)/2 is the same as ceiling(x*3/2).
}

Because of how I calculated the minimum (using a kind of manual ceiling function - see the comment in the code), I also directly address the issue that you fixed in https://github.com/Xilmi/OpenXcom/commit/8d96b4092516399b7c57548a4c1412981d1a1443.

Both these changes are quite cosmetic, so you may think "why bother". What I was thinking though is that moving the Max and Min cost calculation to the ResearchRule (rather than having it around the UI generating code of the Basescape/ResearchInfoState) is first of all cleaner and secondly can enable, with very little effort, to actually modify the standard behaviour of rolling +/- 50%.

Imagine that in a mod you could not only define the average project time but also Max and Min. This way you could have some very well scoped projects (where both Max and Min are near to the Average) and some very random projects allowing for any roll range you can possibly come up with. With my changes, it would take quite little effort to have the getMinCost and getMaxCost actually refence some hardcoded values from the mod, as an option.

Would this be something useful / interesting? Cheers!