Fractal Softworks Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: [TUTORIAL] Semantic Versioning for mods  (Read 555 times)

Jaghaimo

  • Commander
  • ***
  • Posts: 113
    • View Profile
[TUTORIAL] Semantic Versioning for mods
« on: October 22, 2020, 03:19:31 AM »

PSA: Using Semantic Versioning makes it obvious if release is save-game compatible or not

Every release manager has to decide on, or come up with, a versioning scheme. All versioning schemes can achieve the desired result, which is to indicate which version is newer. On the other hand, only some of versioning schemes allow to convey much information. This thread deals with one of such schemes - Semantic Versioning.

In an essence, SemVer compliant version consist of 3 numbers: X.Y.Z (Major.Minor.Patch). From SemVer website:

Quote
Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.

To translate this into more mod-friendly language:
  • Unless your mod is a library mod (e.g. LazyLib, GraphicsLib) all releases that only fix bugs and are save-game compatible increment Patch version. If it is a library, you need to take the API-breaking rule into consideration.
  • Any release that introduces new features and is save-game compatible increments Minor version.
  • Any release that is not save-game compatible increments Major version.

These are the rules for when you HAVE TO increment a given number. They are not the REQUIREMENT to increment a given number though. For example, you may increase Major even if your update is backward compatible, say when you release a big content update.

The immediate benefit of following SemVer is thus obvious - as a modder you no longer need to tell if the release is save-game compatible or not. As a mod user - you no longer need to ask. Other benefits include predictability (X something big, Y something new, Z something fixed), stability (X can break, Y can change, Z is the same), and quality enforcement (very implicit benefit, if you are required to increment major version for every breaking change you may feel compelled to make a non-breaking change instead, or even provide migration code).

Finally, do note that SemVer is not a panacea for everything. Following it comes with both benefits and costs - some mods may implement it with low effort, while for some the implementation cost might not be worth the benefits it brings.
« Last Edit: November 17, 2020, 01:56:15 AM by Jaghaimo »
Logged
Stellar Logistics - courier services
Hyperspace Network - information services

Jaghaimo

  • Commander
  • ***
  • Posts: 113
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #1 on: October 22, 2020, 03:22:25 AM »

Special cases

Public release candidate (rc) - release candidate is, as the name implies a candidate for release. When you release an RC (x.y.z-rc1) you freeze your codebase at this point. Only bugfixes may be added, at which point you release another RC (x.y.z-rc2). Once you are happy with your mod (usually when no new bugs show up during your testing window), you re-release THE SAME RC, just without rc suffix (x.y.z). No changes between last RC and your release should happen - if they did, you should release another RC to be retested. This ensures quality of your mod as it reduces regression bugs (bugs you may have introduced by changing code to fix other bugs / add new features).

Internal release candidate - if your team is large you may want to opt for internal testing and bug fixing cycle. To track iterations you did you may want to use Patch version, there's no rule preventing you from releasing X.Y.21 version as first public version.

Pre-release (pre) - similar to RC, but you can change whatever you want (new features, bugfixes). If it is backward compatible then you would release X.Y+1.0-pre1, if not X+1.0.0-pre1. The purpose of these releases is to gather feedback about new functionality, and it is still during development of new features. If you are ready for release - use RCs or just release.

Alpha release (0.x.x) - SemVer rules do not apply here. All is up in the air, you may increment Patch version and break compatibility. API is considered unstable and may change between any version.
« Last Edit: October 22, 2020, 03:34:37 AM by Jaghaimo »
Logged
Stellar Logistics - courier services
Hyperspace Network - information services

Tartiflette

  • Admiral
  • *****
  • Posts: 3016
  • Toss a coin to your Modder, O' valley of plenty
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #2 on: October 22, 2020, 05:06:55 AM »

Given that usually an update save-compatibility is written in big red text that nobody reads, I find it dubious to posit that a versioning scheme specific to starsector mods would improve anything.

Now having a proper versioning scheme is a good thing though. But I don't think it would be of any use for that specific purpose.
« Last Edit: October 22, 2020, 06:12:05 AM by Tartiflette »
Logged
 

Nia Tahl

  • Captain
  • ****
  • Posts: 459
  • AI in disguise
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #3 on: October 22, 2020, 05:10:36 AM »

Going by whether something is save-compatible or not, doesn't really work that well in practice, either. Very minor changes can cause save-incompatibility while you can have massive updates that do not break saves. Save-compatibility is simply tied to very particular specifics and pretty much completely removed from how major an update is.
Logged
My mods: Sylphon RnD - Tahlan Shipworks - ScalarTech Solutions - Trailer Moments
It's all in the presentation

Tartiflette

  • Admiral
  • *****
  • Posts: 3016
  • Toss a coin to your Modder, O' valley of plenty
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #4 on: October 22, 2020, 05:25:10 AM »

And indeed it could lead to oddities like incrementing a minor version for a massive update but having to increase the major version for the next hotfix.
Logged
 

Jaghaimo

  • Commander
  • ***
  • Posts: 113
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #5 on: October 22, 2020, 06:27:18 AM »

Given that usually an update save-compatibility is written in big red text that nobody reads, I find it dubious to posit that a versioning scheme specific to starsector mods would improve anything.

Not that having a proper versioning scheme is a bad thing though, it's good. Just not for that purpose.
Perhaps I stressed too much regarding save-game compatibility. There are some implicit benefits from following SemVer - namely predictability (X something big, Y something new, Z something fixed), stability (X can break, Y can change, Z is same), and quality enforcement (doubly implicit, if you are required to bump major for every breaking change you may feel compelled to make a non-breaking change instead, or provide migration).
Going by whether something is save-compatible or not, doesn't really work that well in practice, either. Very minor changes can cause save-incompatibility while you can have massive updates that do not break saves. Save-compatibility is simply tied to very particular specifics and pretty much completely removed from how major an update is.
What is a major release? Rewrite? Big new content? It is arbitrary. Under SemVer it only becomes mandatory to increment Major version when breaking compatibility (save game, API, etc) but it is not obligatory to break compatibility in order to increment Major version (you can still increment it if you are releasing something big). With a major update though you are not expected to guarantee backward compatibility.

And indeed it could lead to oddities like incrementing a minor version for a massive update but having to increase the major version for the next hotfix.
A hotfix that is mandatory should always be backward compatible. As it usually means adding extra code, it would probably be better to break SemVer rules: release a patch increment (say 1.10.1) that replaces broken release (1.10.0) and consider it backward compatible (like the broken one never existed).

You could treat SemVer as just guidelines. Following them gives you some benefits but comes at certain expenses. If it's too expensive to follow them - break them when needed.
Logged
Stellar Logistics - courier services
Hyperspace Network - information services

MrFluffster

  • Ensign
  • *
  • Posts: 38
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #6 on: October 22, 2020, 06:59:38 AM »

As a modder you no longer need to tell if the release is save-game compatible or not. As a mod user - you no longer need to ask.

I already talked a bunch about this on the discord but I feel I really need to stress this in a more public space: this is genuinely bad advice. You need to communicate save incompatibility very loudly in the patch, and expecting a user to have additional knowledge to tell that by looking at version is extra extra bad for anyone new to the game.

In general I think that SemVer is simply meant to fix problems that don't really exist in our environment and has requirements that either greatly increasly required work or make this just look silly, nudging you towards warping your workflow around it...
And sorry for bashing you so much for suggesting it but I really don't want this to make someone end up adopting it, it's just not a good idea for us.
Logged
Author of the somewhat-mediocre Fluff's Ship Pack. Check it out!
Feel free to send me feedback on discord @PMMeCuteBugPhotos#0420

Jaghaimo

  • Commander
  • ***
  • Posts: 113
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #7 on: October 22, 2020, 07:18:26 AM »

As a modder you no longer need to tell if the release is save-game compatible or not. As a mod user - you no longer need to ask.

I already talked a bunch about this on the discord but I feel I really need to stress this in a more public space: this is genuinely bad advice. You need to communicate save incompatibility very loudly in the patch, and expecting a user to have additional knowledge to tell that by looking at version is extra extra bad for anyone new to the game.
Communication is always a good thing. The "no longer need" does not mean you should not. Heck, even if you do mention it multiple times in different posts there will be people asking about it anyway.
In general I think that SemVer is simply meant to fix problems that don't really exist in our environment and has requirements that either greatly increasly required work or make this just look silly, nudging you towards warping your workflow around it...
True, its  purpose is for highly dependant software - mostly to solve GNU Linux dependency hell. I believe it can be successfully used by some mods, thus delivering a benefits and improving those mods (quality, process, etc.).

And sorry for bashing you so much for suggesting it but I really don't want this to make someone end up adopting it, it's just not a good idea for us.
I don't feel like being bashed, it's a healthy discussion. I do not agree with generalisation of "SemVer is not good for mods period". Some mods can successfully use it and benefit from using it. For some it will be not be easily applicable.

I've updated the OP to indicate those reflections and outcome of prior discussions we had.
« Last Edit: October 22, 2020, 07:29:31 AM by Jaghaimo »
Logged
Stellar Logistics - courier services
Hyperspace Network - information services

Harmful Mechanic

  • Admiral
  • *****
  • Posts: 1083
  • For space is wide and good friends are too few.
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #8 on: October 22, 2020, 10:31:49 AM »

Yeah, this is ludicrously out of touch for a community where you can write a comprehensive FAQ, voluminous changelogs, label your save-breaking updates and explain the architecture issues that make save-breaking updates a fact of life at length, and users will barely learn to tell which mod contains which factions, systems, ship, weapon, or feature.

Semantic Versioning makes the most sense in a context where you control architecture and users have incentives to learn and consistently apply your versioning. Neither applies to Starsector modding.
« Last Edit: October 22, 2020, 10:33:35 AM by Harmful Mechanic »
Logged
People need societies, but they don't necessarily need nations.

SCC

  • Admiral
  • *****
  • Posts: 2073
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #9 on: October 22, 2020, 11:43:35 AM »

Perhaps I stressed too much regarding save-game compatibility. There are some implicit benefits from following SemVer - namely predictability (X something big, Y something new, Z something fixed), stability (X can break, Y can change, Z is same), and quality enforcement (doubly implicit, if you are required to bump major for every breaking change you may feel compelled to make a non-breaking change instead, or provide migration).
If X is used both for big, but not save breaking updates and for save breaking updates of any size, it kinda loses clarity, since you no longer can tell if a mod is save breaking by looking at X and if any given X update is big. Of course, you can reply that the modder can make a non-breaking change instead or provide migration, but that's expecting a lot of effort and potential future issues for someone who makes mods for free.

Nick XR

  • Captain
  • ****
  • Posts: 428
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #10 on: October 22, 2020, 12:22:10 PM »

Semantic Versioning makes the most sense in a context where you control architecture and users have incentives to learn and consistently apply your versioning. Neither applies to Starsector modding.

Perfectly said. 
The failure modes I see constantly:
  • Not save compatible
  • Not StarSector version compatible

IMO those are the problems we need to fix.  It would be great if the launcher could read a Mod's JSON file and saves to figure that stuff out (work for @Alex).  It would save everyone a lot of wasted effort.

Jaghaimo

  • Commander
  • ***
  • Posts: 113
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #11 on: October 22, 2020, 01:31:53 PM »

If X is used both for big, but not save breaking updates and for save breaking updates of any size, it kinda loses clarity, since you no longer can tell if a mod is save breaking by looking at X and if any given X update is big.
It's not something I'd advocate to do, but is possible if you wish. Major updates in well established projects do come with backward incompatibilities (mostly because it's the only window to remove deprecated code). Even in a small codebase like mod, I will use this possibility to remove deprecated code.

Of course, you can reply that the modder can make a non-breaking change instead or provide migration, but that's expecting a lot of effort and potential future issues for someone who makes mods for free.
Yet this is exactly what the whole open source does, for free. If your update is save-breaking and you don't want to bother with writing migration code just increase the major version. Even if you use SemVer nobody is forcing you to spend extra effort on user experience (which save compatibility is, ability to enjoy new things in existing save - nice to have, not essential).

Perfectly said. 
The failure modes I see constantly:
  • Not save compatible
  • Not StarSector version compatible
IMO those are the problems we need to fix.  It would be great if the launcher could read a Mod's JSON file and saves to figure that stuff out (work for @Alex).  It would save everyone a lot of wasted effort.
But to do so it needs to understand which update is breaking the save. So if not SemVer, then an equivalent that provides versioning regime will be needed. Alex already did this for mod dependencies - this is the same approach as SemVer:
Quote
"version":"0.3.2.1",
Version of this mod; required. Can also be specified as:
"version":{"major":3, "minor":2, "patch":1}
A major version mismatch between the mod and another mod that depends on that version of it means the mod that depends on it can not be enabled. A minor/patch mismatch results in a warning.
All we need to make our life easier is to extend that to save games (refuse to load game if the major changed).

Source: https://fractalsoftworks.com/forum/index.php?topic=4760.msg258976#msg258976
« Last Edit: October 22, 2020, 01:45:48 PM by Jaghaimo »
Logged
Stellar Logistics - courier services
Hyperspace Network - information services

Nick XR

  • Captain
  • ****
  • Posts: 428
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #12 on: October 22, 2020, 02:58:24 PM »

Perfectly said. 
The failure modes I see constantly:
  • Not save compatible
  • Not StarSector version compatible
IMO those are the problems we need to fix.  It would be great if the launcher could read a Mod's JSON file and saves to figure that stuff out (work for @Alex).  It would save everyone a lot of wasted effort.
But to do so it needs to understand which update is breaking the save. So if not SemVer, then an equivalent that provides versioning regime will be needed. Alex already did this for mod dependencies - this is the same approach as SemVer:

I too am good with what ever Alex comes up with, he just needs to by fiat enforce something. (especially the "don't load mods that aren't built for this version" problem)
Also I think much of the blame lies with VersionChecker encouraging people to update their stuff when the updates it suggests might very well brick their save.  Further the fact that version incompatibility warnings happen after you update a bunch of mods makes it difficult to revert to the prior good state (for most users).


I personally feel the only safe and sane strategy is to update your mods before you start a new play through. Everything else is just asking for trouble.

Morrokain

  • Admiral
  • *****
  • Posts: 1518
  • Megalith Dreadnought - Archean Order
    • View Profile
Re: [Guide] Semantic Versioning for mods
« Reply #13 on: October 26, 2020, 02:54:24 PM »

I get that standardizing the versioning would remove a lot of headaches for experienced modders. It is really annoying when you spend the time posting details and a user disregards them, slaps a million mods on, and gets mad and flame posts that your mod doesn't work, the game is buggy, etc. I think we can all relate to that being frustrating when you have spent so much time on your work.

I'm going to have to echo what many said here, though, and say I don't think strict version management like this is good for the community as a whole. Standardizing the versioning is fine and I think the guide is very helpful, but having additional software purposefully lock out a game load or cause a well-meaning CTD creates a larger barrier of entry to would-be modders and even more unrealistic expectations of end users.

Why? A new modder has to have intimate knowledge of what causes saves to break. That is definitely a learn-as-you-go kind of thing and as the code base of a project expands, more things can go wrong in that regard. So they have two choices:

A) Release every version of their mod as a Major release so they protect themselves from additional flak from end users - since there would then be "no excuse" for a save-breaking update as a Minor or Patch update from the perspective of the end user.

B) Release the actual version of their mod considering the added content/etc, and risk being wrong and the save being broken. In that case, again, end users now have higher expectations and therefore greater wrath than before for mistakes. That point also goes (possibly doubly so) for experienced modders who make a mistake - which will likely happen from time to time.

Option B's downsides are self evident. Option A's downsides are a lack of clarity on actual content contained in a Major update. What this would likely translate to would be: experienced modders generally follow the guidelines (except then they don't! - because they know a hotfix breaks a save, etc) and release a lot of additional content under the Major update. New modders will be all over the place, and the frustration of the end user when a problem occurs will be greater - not less - for situations where a save is broken because they had a false illusion of safety. Those situations may happen less as a whole, true, but it is skewed in the benefit of those with experience - as would be the Major update in general since more experienced modders can take better advantage of it and bring more consistent results (that end users now expect!) each release.

So, imo, even though the desire for SemVer enforcement is well-meaning, it doesn't *really* solve the problem at the end of the day. It ends up making things more confusing for everyone, end user included, and makes a new modder's job even harder.

SemVer is great for tech companies and technical teams to manage their version control and choose release candidates for low-margin-of-error production environments, but it really has no business being enforced in a hobbyist modding scene. When a mod update breaks saves, it may annoy some end users, but its not like you are going to have to explain to a project manager why the release went so bad. It's a hobby you are doing for free.

Now, all of that being said, something like what was proposed here that would allow a modder, themself, to create CTDs based upon versioning within their own mod - and not globally to all mods - would be ok. That way, a modder can opt in to the additional pressure of increasing user expectations by controlling their mod's launch-ability when encountering mismatching versions. Unfortunately, that will still create situations where an end user posts on a newbie modder's thread wondering why their mod doesn't police its versioning like X mod does, but that is at least better than a universal standard of expectation for releases as would be found in a professional environment.
Logged