Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Jaghaimo]

The important thing here is consistency and user expectation though.

I would also add user experience and value to business (modder in this case).

User experience (as a user) - being consistent and meeting expectations. My experience is good when I upgrade the mod and my game does not break. The reason for upgrading midgame might be serious bugfix, balance fix, or just non-essential new content that I want to try. At the same time, if I upgrade to save-breaking game I want the game NOT to load, as otherwise I will break first statement. Also, I want a clear and concise instructions explaining what to do to make my game loadable.

Business value (as a modder) - I can prevent my users from loading incompatible versions in their saves, thus saving time debugging stack traces and explaining myself in my forum thread. To do so, I want the save-game preventing message to be clear and concise (so the user can follow it without consulting forums), and "IN YOUR FACE" (so that user can't do anything other than dismiss it and follow the instructions).

If that is correct, the big problem which I brought up in the other thread is that the actuality of that information isn't guaranteed to be correct.

I would have to disagree with you on that. This could be said about any software that does not have a full suite regression tests in place. At this point you are trying to accurately infer backward compatibility based on the changes you did to your software and your understanding of its interaction with other software. Best effort is enough.

Going further, it does not need to be correct, it only needs to be enforced correctly. If I mark my new version 1.5.2 as incompatible upgrade even though it might be compatible with 1.4.0+ that's not a big issue (will only prevent users from upgrading mid-game), and something I as a modder can do.

(Tartiflette's separation of versioning and compatibility control is the best way to do this, I'd say, since it removes the additional issue about versioning clarity through always incrementing Major if unsure about save compatibility. But mark my words, it will still open up a can of worms the first time anyone is wrong should it be implemented!  )

This is a huge can of worms. Let's go back to initial requirements: "consistency", "user expectation" and "user experience".

Modder releases a new version, 2.0.0 (internal 23), that comes after 1.4.5 (internal 23), that comes after 1.4.4 (internal 22). User can no longer upgrade from <=1.4.4. If we prevent user from loading the game, what do we show? Please downgrade? User goes back to 1.4.5 (which he skipped). Same problem. We can say "Downgrade to internal build 22 (it also need to be stored in a savegame, because internal might not be simply current - 1). But internal version is being hidden, which makes user go straight back to forum thread, which tells user "open mod_info and find one with 22", which means "keep downloading previous versions until you find one matching" (as internal is not published). Also this does not solve consistency, as you can't tell from version alone which versions are safe to replace current version.

So the solution is to expose internal where you have a version and a internal suffix (at least when posting, to communicate internal, e.g.: 2.0.0+i23, 1.4.5+i23, 1.4.4+i22), or make it into Major. Two viable schemes come to my mind, SemVer again (23.2.0.0, 22.1.4.5, 22.1.4.4, Major.Content.Minor.Patch) and Content before Major: Content.Major.Minor.Patch (which still makes sense for version comparison, 2.23.0.0, 1.23.4.4, 1.22.4.4).

As an alternative to that, beforeGameLoad hook as per OP would do as well (and API that exposes logic Alex already added to handle dependencies and information displayed in incompatibilities tooltip).

[Morrokain]

I would also add user experience and value to business (modder in this case).

User experience (as a user) - being consistent and meeting expectations. My experience is good when I upgrade the mod and my game does not break.

Right, in general I agree, but it wouldn't be the case for everyone because of knowledge requirements for Best Effort. Because of that, the above benefit is illusory to a user in the actual modding scene. Again a "business" has the expectations of quality control that you are speaking to because they have infrastructure dedicated to acquiring talented and knowledgeable staff to interpret and implement the requirements. That is simply not the case here, and so enforcing helps some and hurts some - with it causing the most problems for those just starting out. It certainly brings value to a modder who understands all the requirements, no argument there - it is just that it is disingenuous to assume that all do. That wouldn't be a problem in and of itself, but as those who understand the requirements raise user expectation in this way, those who don't suffer a larger penalty of "user disappointment" that they don't.

If that is correct, the big problem which I brought up in the other thread is that the actuality of that information isn't guaranteed to be correct.

I would have to disagree with you on that. This could be said about any software that does not have a full suite regression tests in place. At this point you are trying to accurately infer backward compatibility based on the changes you did to your software and your understanding of its interaction with other software. Best effort is enough.

Going further, it does not need to be correct, it only needs to be enforced correctly. If I mark my new version 1.5.2 as incompatible upgrade even though it might be compatible with 1.4.0+ that's not a big issue (will only prevent users from upgrading mid-game), and something I as a modder can do.

I really fail to see a difference except the key one: best effort isn't a guarantee. Your example, of course, has no downsides because marking an update incompatible when it is, in fact, compatible leaves the user none the wiser. What about the reverse? You mark it compatible and it is not. Keep in mind, a user won't always know this when trying to load the save. Off the top of my head, they have a ship with a hullmod equipped in storage on some market somewhere and that hullmod was removed in the current (supposedly compatible) update. They won't get a crash until probably interacting with that market, etc. (Not really sure if this is true anymore, but it was at one time.)

I mean, this isn't the worst thing in the world if it happens. My point was that the difference between this happening now and after the suggestion is implemented (to a user) is that it might actually end up being more off-putting because it was clear, concise, and in your face so many times before. That, and because mods that manage the feature well and don't make this mistake cause it to be a rarer occurrence - which again is more likely to effect a new modder than one who understands the game engine and API to the point that they can effectively manage this. I mean, I'm not trying to say it would be awful or ruin the modding scene or anything, but it is certainly something to consider.

(Tartiflette's separation of versioning and compatibility control is the best way to do this, I'd say, since it removes the additional issue about versioning clarity through always incrementing Major if unsure about save compatibility. But mark my words, it will still open up a can of worms the first time anyone is wrong should it be implemented!  )

This is a huge can of worms. Let's go back to initial requirements: "consistency", "user expectation" and "user experience".

Modder releases a new version, 2.0.0 (internal 23), that comes after 1.4.5 (internal 23), that comes after 1.4.4 (internal 22). User can no longer upgrade from <=1.4.4. If we prevent user from loading the game, what do we show? Please downgrade? User goes back to 1.4.5 (which he skipped). Same problem. We can say "Downgrade to internal build 22 (it also need to be stored in a savegame, because internal might not be simply current - 1). But internal version is being hidden, which makes user go straight back to forum thread, which tells user "open mod_info and find one with 22", which means "keep downloading previous versions until you find one matching" (as internal is not published). Also this does not solve consistency, as you can't tell from version alone which versions are safe to replace current version.

  Maybe I'm missing something here? I don't see how having internal exposed in the version, itself, helps make it more clear to the end user what their course of action should be. Like, if you release 23.1.3.4 and one user has 22.4.1.0 and another 23.0.8.5 there is still a lack of clarity what version to revert to unless the version of the save itself is exposed to the user (which it is already, right?). If you are speaking to the idea that the version is stored in the save file and the user can be pointed to it explicitly that way, then why does internal even need to be exposed? It would point to the version the user was last using regardless, and that version is guaranteed to be at the compatible internal level of the save...

I don't think "at a glance" compatibility via the suffix implementation is worth entangling content and compatibility into the same number. That, I believe, was Tartiflette's point. What you at least seem to be thinking of in this case, is when a user asks why their save isn't loading and all they have is their current version number. It would would be useful, in some ways I suppose, to have at-a-glance compatibility then. Even that is a little suspect, come to think of it, because you'd still need to know the version the save is trying to reference - so you're back at square one. But when considering the warning message when trying to load a save? All pertinent information is already available from the perspective of what such a feature is trying to accomplish.

If a user's save is 1.4.4 (internal 22) and the release is 2.0.0 (internal 23), it doesn't matter whether a 1.4.5 (internal 23) exists or not, the message would still point the user back to the save's version (1.4.4 internal 22). Trying to be extra smart and say "Hey there is a 1.4.5 (internal 23) - go try that first!" - wouldn't be possible even if the versions were, respectively: 23.2.0.0 - 22.1.4.5 - 22.1.3.0. The message is still going to be pointing the user to their save's version - 22.1.3.0 even if, via the version control, 22.1.4.5 would technically be compatible.

To do otherwise would require every legacy version of the mod to be stored in a database, and queried to check for "best backward compatibility" each time a mismatch was found. That's extremely unlikely to ever happen.

[Jaghaimo]

Right, in general I agree, but it wouldn't be the case for everyone because of knowledge requirements for Best Effort. Because of that, the above benefit is illusory to a user in the actual modding scene. Again a "business" has the expectations of quality control that you are speaking to because they have infrastructure dedicated to acquiring talented and knowledgeable staff to interpret and implement the requirements. That is simply not the case here, and so enforcing helps some and hurts some - with it causing the most problems for those just starting out. It certainly brings value to a modder who understands all the requirements, no argument there - it is just that it is disingenuous to assume that all do. That wouldn't be a problem in and of itself, but as those who understand the requirements raise user expectation in this way, those who don't suffer a larger penalty of "user disappointment" that they don't.

The various systems we are discussing in this thread are optional, and not enabled by default (only warning will show on tooltip using logic Alex already has).
New modders and/or ones that do not want to learn (improve) should simply ignore the (in whatever form we get, if any) forced save incompatibility.
Without this modders that would want to make their lives easier (and can afford the cost of following whatever is decided) cannot, and like the rest, will have to default to costly "forum thread" support.
Yes, one could argue that "users don't read" and "forum support" will still be a thing, but at least the number of posts per release should drop (because some users do read, it's just now they get stack-trace when mod fails to load (or worse, during the gameplay later)).

I really fail to see a difference except the key one: best effort isn't a guarantee. Your example, of course, has no downsides because marking an update incompatible when it is, in fact, compatible leaves the user none the wiser. What about the reverse? You mark it compatible and it is not. Keep in mind, a user won't always know this when trying to load the save. Off the top of my head, they have a ship with a hullmod equipped in storage on some market somewhere and that hullmod was removed in the current (supposedly compatible) update. They won't get a crash until probably interacting with that market, etc. (Not really sure if this is true anymore, but it was at one time.)

We have two problems here:
* False positives (modder claiming mod is save compatible while it is not), and
* False negatives (modder claiming mod is not save compatible, while it is).

We already discussed the second case - regardless of modders reason to mark update as save incompatible (mistake or deliberate decision) user experience will not suffer, and modder benefits will not be affected.
As for former, mistakes happen. Without thorough testing it is impossible, and even then mistakes will happen.
Experience, and having a test plan that covers most use-cases help (in my case I have a save from all my versions, where I have all assets from my mod on "me" and try to load all saves with the new version).
Again, this being optional - modders that cannot accurately assess the compatibility can decide not to opt to whatever system we get.
Either way they will have false positives and negatives, only this time in a form of a forum thread.

Maybe I'm missing something here? I don't see how having internal exposed in the version, itself, helps make it more clear to the end user what their course of action should be. Like, if you release 23.1.3.4 and one user has 22.4.1.0 and another 23.0.8.5 there is still a lack of clarity what version to revert to unless the version of the save itself is exposed to the user (which it is already, right?). If you are speaking to the idea that the version is stored in the save file and the user can be pointed to it explicitly that way, then why does internal even need to be exposed? It would point to the version the user was last using regardless, and that version is guaranteed to be at the compatible internal level of the save...

User with 22.4.1.0 that upgraded to 23.1.3.4 should revert to latest 22.x.x.x.
User with 23.0.8.5 does not need to revert, as line 23.x.x.x is save compatible with any previous versions in that line.
In both cases we can present user correct version to downgrade (download latest Z.x.x.x, where Z is already stored in a save-game).

If a user's save is 1.4.4 (internal 22) and the release is 2.0.0 (internal 23), it doesn't matter whether a 1.4.5 (internal 23) exists or not, the message would still point the user back to the save's version (1.4.4 internal 22). Trying to be extra smart and say "Hey there is a 1.4.5 (internal 23) - go try that first!" - wouldn't be possible even if the versions were, respectively: 23.2.0.0 - 22.1.4.5 - 22.1.3.0. The message is still going to be pointing the user to their save's version - 22.1.3.0 even if, via the version control, 22.1.4.5 would technically be compatible.

"the message would still point the user back to the save's version (1.4.4 internal 22)." the message could and only would point to the version the save game uses. There could be 1.4.4.1 (22) but we could not advise the player to try 1.4.4.1, nor mention hidden internal 22. Only with exposing internal could we show user which line is compatible (use latest with suffix -22).

"Hey there is a 1.4.5 (internal 23) - go try that first!" is wrong, user is on 1.4.4 (internal 22) so he can only use internal 22 (1.4.4).Also, same as above, we can get Major from save and tell user to user downgrade to the latest Major release.

Let's shift a discussion a bit:

I don't think "at a glance" compatibility via the suffix implementation is worth entangling content and compatibility into the same number. That, I believe, was Tartiflette's point. What you at least seem to be thinking of in this case, is when a user asks why their save isn't loading and all they have is their current version number. It would would be useful, in some ways I suppose, to have at-a-glance compatibility then. Even that is a little suspect, come to think of it, because you'd still need to know the version the save is trying to reference - so you're back at square one. But when considering the warning message when trying to load a save? All pertinent information is already available from the perspective of what such a feature is trying to accomplish.

The essence of Tartiflette's internal is: hide Major (from SemVer point of view). So now we have Content.Minor.Patch, with Major (internal).
SemVer applies, but we have no idea what the Major is and what versions are compatible.

The essence of my "expose internal" is to bring back Major to Version, but consider it being elsewhere so that modders desire to be the first number a Content (large updates) be met.
Thus few explorations: Content.Minor.Patch-Major, Content.Major.Minor.Patch, Major.Content.Minor.Patch.Last option makes most sense as we take first part of version (but Content is not the king here), first option is second best (as for compatibility we take first part).

None of this by default would enforce save loading prevention (only when mod opts in), but all of this would (by having Major visible) direct user to correct version to downgrade to.

[Morrokain]

Re: False Positives and False Negatives

Yes I understand that and why it is appealing and that it would save someone (including myself) some extra time, potentially. I'm talking about the psychology of user feedback. Making the lives easier of experienced modders inherently widens the gap of what is overall acceptable to the average user. Sad but true.

Let me put it another way. You have 3 restaurants. 2 of the 3 make a mistake 10/100 times and automatically refund your meal, no questions asked, for 7 of those 10 mistakes without a trip back to the restaurant. The 3rd restaurant makes a mistake 35/100 times and you have to return your food with a receipt within 30 minutes of pickup - untouched - for a refund.

Which restaurant are you likely to get food from?

I'm not against meritocracy or anything, but the odds of restaurant 3 going out of business (a.k.a a new modder sees the quality and automation gap between their mod and others and is overwhelmed) are a lot higher with a lot of these kinds of automation in place. The more complex or the higher amount of automation "plugs" required for QOL features, the less likely they will all get picked up and the worse those who do not pick them up look.

Now I want to be clear that the above reason isn't justification enough to not make our lives as modders easier sometimes. It's just... well a very real side-effect. So, I would caution against going overboard with it. I'm also not saying that this particular suggestion is overboard and I know it's opt-in and all that, but I did want to bring up this downside as a part of the discussion.

(I think modders who are used to a professional development setting often want to utilize those principles and have the same QOL support as they would have there... but you have to be careful because this isn't a professional setting and to make it seem like one will kill the desire of the hobbyist to get involved. I can't speak for everyone, but I only got the degree I did because I started modding as a hobby a long time ago. I chose Starsector, not just because I could see the game's potential and boy was I right! - but also for the very reason I bring up - that it had an "ease of access and expectation" when considering other moddable games. In short: it was one of the easiest games I could find to mod at the time. Maybe this makes me foolish as I get that technical considerations like compiling code into a jar, etc, become necessary in the long haul. As much as I understand that, I still try and do my best to keep it accessible when stuff like this comes up.)

"Hey there is a 1.4.5 (internal 23) - go try that first!" is wrong, user is on 1.4.4 (internal 22) so he can only use internal 22 (1.4.4).Also, same as above, we can get Major from save and tell user to user downgrade to the latest Major release.

Whoops - typo! I meant 1.4.5 (internal 22) - so compatible with any internal 22 release but not the 2.0.0 (internal 23) release.

But anyway:

User with 22.4.1.0 that upgraded to 23.1.3.4 should revert to latest 22.x.x.x.
User with 23.0.8.5 does not need to revert, as line 23.x.x.x is save compatible with any previous versions in that line.
In both cases we can present user correct version to downgrade (download latest Z.x.x.x, where Z is already stored in a save-game).

This is what I mean by "too smart" though. You are wanting to use the version in the save to allow for parsing the compatibility component (hence exposing Internal/Major as compatibility) in order to use logic to determine "best backward compatibility" (so as you say - detect the Z from Z.x.x.x or x.x.x.Z or x.Z.x.x, etc)

Alternatively, instead of through code logic on save load, if a user does ask about it, (so the primary purpose of version control to avoid thread support has failed, as it were) - then you at least have a frame of reference (if they know the save version, mind you, as the app version doesn't matter) to say "latest" Z.x.x.x update should be fine to use.

Yeah I can definitely see the appeal.

My point is that, while admirable, it would be far simpler to just forget about Z altogether, and use the launcher's logic to simply display the version from the save - and recommend that - instead of the extra step of trying to find a better/newer version with the same compatibility level - "Z". That way, Z doesn't need to be in the version and won't get confused with content (especially because most games version themselves around content not compatibility).

The benefit to what I will call "finding the best Z" as you are describing, is that it can potentially allow the addition of some new minor content or bugfixes during midgame for the user. The downside? If you are wrong somewhere along the line, especially if you are not made immediately aware of that fact (think of the hullmod example) then the additional complexity is pointless and more confusing. Since you are correct that mistakes will happen even with intensive testing, I'd say simple is best here.

For instance, I'd hate if a user was on bug-free stable version 23.1.4.5 and came back to the game after a while and updated to 24.2.5.3. The message would say "Revert to the latest 23.x.x.x version" and they would see that 23.2.5.1 was the latest in this case. I think it is unwise to assume that 23.2.5.1 would be the best reversion. It could have additional bugs that were fixed in the first 24.0.0.0 version but weren't present in 23.1.4.5 (yes I get that is a mistake of the modder to not patch 23.2.5.1 to 23.2.5.2 but trust me it can happen with large projects - cutting a release takes time and sometimes you just want keep moving on with development), or worse, be one of those updates that slipped through the compatibility cracks and breaks the save further down the line.

That's a headache for everyone when the simpler solution was to just paste the original save's version in the message. Errr, is this making sense? lol It's hard to translate brain to text sometimes and I'm fairly sleepy atm. 

Anyway, good discussion and thank you for being courteous!

*EDIT*

That's a headache for everyone when the simpler solution was to just paste the original save's version in the message. Errr, is this making sense? lol It's hard to translate brain to text sometimes and I'm fairly sleepy atm. 

I reread this and just in case it came across as sarcastic: I'm genuinely trying to think of ways that this kind of feature can backfire.

[Morrokain]

*delete please*

[Jaghaimo]

I get all of what you are saying, and agree with you. Retaining the low entry level is paramount to maintaining healthy modding community. But so is improving the quality (however defined) of those mods. Adding a new quality measure (backwards compatibility and/or backwards compatibility claim accuracy) would not fundamentally shift position of challenging modders. It would force behemoths to adapt though, but once those adapted it would be one more obstacle in gaining a foothold.

For example, new faction mods have to compete with existing behemoths on multiple levels: sprites, balance, lore, fun factor - just to name few. As they evolve, their creators learn and improve, and at one point their mod joins those behemoths. If we started considering other qualities like code quality (forcing modder to improve his general coding skills), processes (improve development, regression testing, or release processes), standards (modder conforms to well established standards), creators would once again adapt, and we would end up with a healthier modiverse.

We already have some of those (see: vanilla-balance guidelines, polygon granularity guide). Adapting a versioning scheme or agreeing on a backward-compatibility declaration (via mod_info.json fields, versions, thread topic which we already do) is another such. And a step to improve general quality.


Enough off topic, back to the problem at hand


If we take the most regressive behaviour - take the exact version from save when telling the user to downgrade - we will save ourselves from a lot of pain as we are effectively blocking mid-save upgrades. What about bugfixes though? SemVer tries to address that with Major, Minor, Patch, but we agreed already that SemVer will not work for the majority of modders, so we can't enforce behaviour based on version alone.
Let's forget about versioning schemes for a moment - how would we allow for an exception (because bugfix, patch) in the system that prevents version change mid-save? Is the "beforeGameLoad" the only alternative? Same version but optional suffixes (-patch1, -patch2, -hotfix, etc.)? I feel that whatever we come up with here will suffer from the same problem: false expectations. If I see mod being "save compatible" I will be disappointed exactly the same regardless of where the information came from (mod thread like now, through mod_info claiming to be valid upgrade, SemVer, onBeforeLoad).

In short: it was one of the easiest games I could find to mod at the time.

For me it was a game I could create the most complicated mods at the time. There are not many games where you can create mods in the same language (usually you are given some scripting language or Lua) as the game (or even other languages, like Kotlin), has proper API exposed, and the only ceiling is your imagination (and API to an extent). At the same time, as I was musing to my wife a while back, it is the only game that I know that has so low level entry barrier to modding. There are people that try to create mods (scripts included) that have no idea about Java or haven't seen an IDE in their live. It's beautiful.

That's a headache for everyone when the simpler solution was to just paste the original save's version in the message. Errr, is this making sense? lol It's hard to translate brain to text sometimes and I'm fairly sleepy atm. 

Most often the simplest solutions are the best ones in the long run. Prevent any upgrade mid-game, force downgrade to the exact version used before... But what about hotfixes/patches then? As a modder I can always re-release my mod with a hotfix, and keep the old version in modinfo to workaround version mismatch prevention. But I can make a mistake and make it save incompatible - back to square one.

I reread this and just in case it came across as sarcastic: I'm genuinely trying to think of ways that this kind of feature can backfire

None at all, I do appreciate all the edge cases and reservations you raise. Without them, we couldn't try to understand the (larger) problem.