Fractal Softworks Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Station Balance - Methodical Analysis  (Read 11337 times)

Dark.Revenant

  • Admiral
  • *****
  • Posts: 2702
    • View Profile
    • Sc2Mafia
    • Email
Station Balance - Methodical Analysis
« on: July 31, 2019, 01:38:38 AM »

As I personally have never created a station in Starsector before, and yet am endeavoring to do so, I reasoned that I need a repeatable test procedure to measure stations.  This would help me determine the overall strength of the station, at least as a reasonable preliminary balance point.

After creating a test-bench and a methodology to go along with it, I began measuring the vanilla stations to establish a baseline to judge my own creations upon.  This led me to some interesting discoveries that, in some cases, confirm "metagame" assumptions about the stations.  In other cases, however, the test results directly contradict accepted "metagame" knowledge.

Test Method
  • Use a battle size of 500, and all other gameplay settings (such as max ships per fleet) are left at vanilla defaults.
  • Use the mods Ship and Weapon Pack, Underworld, Interstellar Imperium, and any applicable faction mod required to spawn the station being tested.
    • The rationale for this is simple: my goal for this was ultimately to make Imperium stations, so that mod contains the testbench and is included by default.
    • In addition, SWP is required for the procedural faction framework I used.
    • Since the Imperium and SWP mods are already included, I added Underworld because just about everyone who is playing with the former two mods will be using UW as well.
  • Spawn a 100%-CR station, no officer, no auto-fit, for the player side and assign full control of the fleet and station to the AI.
  • Generate a procedural faction with unweighted access to the ships, weapons, fighters, and hull mods available to all other factions (including derelicts and remnants) - using the following doctrine:
    • Warships 3 / Carriers 2 / Phase Ships 2
    • Officers 2 / Ship Quality 3 / Fleet Size 2
    • Ship Size 3
    • Aggression 3
    • Combat Freighter Fraction 1.0
    • Auto-fit Randomize 0.1
  • Select a fleet size, in FP (Fleet Points) according to the following logic:
    • If this is the first test, begin at the best guess of the tester (e.g. 100 FP for an orbital station).
    • Otherwise, adjust by some constant number of FP (typically 5% of the original guess): up, if the station won the last bout; down, if the station lost the last bout.
  • Spawn a 70%-CR fleet of the faction and size described above, no officers, using auto-fit, no D-mods, for the enemy side - and use starting command points equal to the fleet size divided by 40, rounding down.
  • Repeat the previous step up to 1,000 times if the fleet's size does not match the target size, choosing the single fleet that wound up closest to what we want.
  • Generate a random battle scenario using the same algorithms/parameters as a campaign station battle.
    • Non-hyperspace.
    • 50% chance of being in a nebula.
    • 48.8% chance of being in an asteroid field; 1-20 asteroids nearby if so.
    • 43.8% chance of being in at least one ring system; 6.3% chance of being in two ring systems.
    • Fixed standoff range of 6000 units.
  • The enemy is not allowed to retreat, must fight to the last, and must deploy all ships.
  • Repeat the test until a minimum of 10 bouts of mixed or unpredictable outcomes (i.e. a mix of Win and Lose) has been observed.
  • The final score is the sum of the fleet sizes of the last 9 bouts plus the fleet size the following bout would have been (had the test continued), divided by 10, rounded to a whole number.

The generated scenario isn't meant to be perfectly realistic to a particular in-game scenario, but rather something that won't be unfairly biased towards one particular fleet composition over another.  The following represents a typical max-strength fleet:


To speed up testing, a plugin automatically speeds the game up such that the internal game logic is running at 1/30 second intervals at the fastest possible speed, limited by the refresh rate and system hardware.

Results
Low Tech Orbital Station: 124 FP
Midline Orbital Station: 135 FP
High Tech Orbital Station: 161 FP

Midline Battlestation: 207 FP
High Tech Battlestation: 245 FP
Low Tech Battlestation: 253 FP

Low Tech Star Fortress: 346 FP
High Tech Star Fortress: 354 FP
Midline Star Fortress: 358 FP

Damaged Remnant Station: 173 FP
Remnant Station: 370 FP

Shadowyards Orbital Station: 170 FP
Shadowyards Battlestation: 333 FP
Shadowyards Star Fortress: 394 FP

OCI Orbital Station: 205 FP
OCI Battlestation: 490 FP
OCI Star Fortress: 579+ FP

Note: the OCI Star Fortress is literally off the chart.  I couldn't generate fleets above ~600 FP, as the test method essentially broke down and couldn't scale high enough to produce an accurate score.

Analysis
Aside from the Remnant Stations and the High Tech Orbital Station, the tested FP scores are not that far off from the advertised FP scores in the game files.

As for balance between the tech levels, at least when it comes to a station soloing an AI fleet, the star fortresses are all basically balanced with each other.  As for the battlestations, Midline is a clear loser - perhaps because of the lack of armor and dissipation on the main module.  For the orbital stations, High Tech is the runaway winner thanks to how strong those shields are.

The big surprises are:
  • Midline is strongly competitive at tier 3, even though it is lagging behind at tiers 1-2.
  • High Tech is not utterly dominant (other than at tier 1).
  • Low Tech is strongly competitive at tiers 2-3.

However, let us not forget a few important facts about the vanilla stations that significantly color the overall experience players have with them, observed through gameplay and many hours of looking at AI tests:
  • Low Tech stations tend to incite AI glitches at an alarming rate, particularly with regards to shooting through/at invincible modules, wasting ammo and flux to achieve nothing.
  • Midline stations are disproportionately easy to dispatch by a clever player by taking out the protective spurs and sending precise shots through the shield gap towards the main broadside section.
  • High Tech stations are reportedly frustrating and dangerous to fight because of how their shields work and of the ship-loss risk the mine strike modules pose.

If we assume that 60 FP with this test methodology is a "level 1" threat (on a scale from 0 to 10), we have the following threat classifications:
Orbital Station: Level 2
Damaged Remnant Station: Level 3
Battlestation: Level 4
Star Fortress: Level 6
Remnant Station: Level 6

A level 0 threat is a truly beginner challenge solvable by the tutorial fleet, while level 10 represents a challenge that requires game mastery to overcome, because it isn't possible to overcome through sheer numbers.  While I think a level 20 officer on a station affects the station more than a bunch of level 20 officers in the player fleet affect the player's fleet (considering you're boosting 100% of the station and perhaps 37% of the player's fleet), there is definitely still room to include even tougher threats in the campaign world.
Logged

Alex

  • Administrator
  • Admiral
  • *****
  • Posts: 19441
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #1 on: July 31, 2019, 09:07:53 AM »

Very interesting!

(In case it's of interest: the approach I used when balancing these was *mostly* to have them face off against a fixed set of capital ships (iirc, Conquest, 2x Onslaught, Paragon) to see how it goes, and tuning from there. Of course there was a bunch of other testing, as well, but that was the starting point.

A possible additional consideration: I think the stations may benefit from support - especially player support - to varying degrees. In particular the midline one; it's got the most pronounced windows of vulnerability, so it seems probable that when deployed with the player - or even with some AI support - it would benefit more than the other stations. It does seem like it'd be the most vulnerable one solo, vs a swarm of enemies.

Hmm. It seems to make sense that the battlestation tier is where it'd be most disadvantaged... at the star fortress tier, it's got armor plating and drone/mine support, while at the orbital station tier, well, it's just a lot of firepower for that tier; its "support"-type modules added by the higher tiers are comparatively weaker than the "citadels" of the other types and more of its power is in the main module.

I'll say, though, the trickiest part of making these, for me, was coming up with ways to make them feel distinct from each other in terms of gameplay, both attacking and defending the stations.)
Logged

intrinsic_parity

  • Admiral
  • *****
  • Posts: 2437
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #2 on: July 31, 2019, 10:50:07 AM »

How does the battle station simulation work in game when the player isn't around? Is there just some formula that figures out if the station wins or not based on deployment points and stuff?
Logged

SCC

  • Admiral
  • *****
  • Posts: 3196
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #3 on: July 31, 2019, 11:01:58 AM »

Autoresolve takes fleet points (value representing a rough estimate of a given ship's combat worth, currently invisible in the game), officers and d-mods into consideration. It's simulated enough that individual ships can take damage and stations can lose modules.

Thaago

  • Global Moderator
  • Admiral
  • *****
  • Posts: 5876
  • Harpoon Affectionado
    • View Profile
    • Email
Re: Station Balance - Methodical Analysis
« Reply #4 on: July 31, 2019, 01:01:13 PM »

Very interesting results! And the testing methodology looks like a very sound middle ground as well. Nice job DR :D. Can't wait to see what kind of stations you come up with.
Logged

intrinsic_parity

  • Admiral
  • *****
  • Posts: 2437
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #5 on: July 31, 2019, 02:40:01 PM »

I'm just curious if there are any differences between the stations in auto resolve
Logged

Wyvern

  • Admiral
  • *****
  • Posts: 2931
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #6 on: July 31, 2019, 04:58:06 PM »

Interesting!  Neat data.  Will say I'm surprised the Shadowyards station scores so much higher than the vanilla ones; they didn't feel notably stronger to me in gameplay... but I guess that's what testing is for.

And now I'm curious what this "OCI" station is; I don't offhand recognize that TLA*, and I wasn't aware that anyone other than MShadowy had gotten a properly implemented station into the game.

* Three Letter Acronym.
Logged
Wyvern is 100% correct about the math.

Morgan Rue

  • Commander
  • ***
  • Posts: 137
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #7 on: July 31, 2019, 08:58:25 PM »

OCI is a [REDACTED] which is not yet finished enough for the forums. It has a station though.
Logged
Dauntless.

dis astranagant

  • Lieutenant
  • **
  • Posts: 56
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #8 on: August 05, 2019, 01:03:21 AM »

Artefact has a couple factions with stations.  Player buildable ones even.
Logged

Tackywheat1

  • Commander
  • ***
  • Posts: 190
  • Paragon is Perfection
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #9 on: October 27, 2019, 02:26:21 PM »

I agree with everything except the remnant station... I always felt like they were more on par with the high tech battlestation rather than a star fort, although maybe it is because they lack the mines
Logged

majorfreak

  • Commander
  • ***
  • Posts: 228
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #10 on: October 27, 2019, 02:42:59 PM »

be neat to make stations have more PD. my usual tactic right now is just to spam pilums. pop
Logged

Morrokain

  • Admiral
  • *****
  • Posts: 1978
  • Megalith Dreadnought - Archean Order
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #11 on: October 28, 2019, 12:29:40 AM »

Very interesting!

(In case it's of interest: the approach I used when balancing these was *mostly* to have them face off against a fixed set of capital ships (iirc, Conquest, 2x Onslaught, Paragon) to see how it goes, and tuning from there. Of course there was a bunch of other testing, as well, but that was the starting point.

A possible additional consideration: I think the stations may benefit from support - especially player support - to varying degrees. In particular the midline one; it's got the most pronounced windows of vulnerability, so it seems probable that when deployed with the player - or even with some AI support - it would benefit more than the other stations. It does seem like it'd be the most vulnerable one solo, vs a swarm of enemies.

Hmm. It seems to make sense that the battlestation tier is where it'd be most disadvantaged... at the star fortress tier, it's got armor plating and drone/mine support, while at the orbital station tier, well, it's just a lot of firepower for that tier; its "support"-type modules added by the higher tiers are comparatively weaker than the "citadels" of the other types and more of its power is in the main module.

I'll say, though, the trickiest part of making these, for me, was coming up with ways to make them feel distinct from each other in terms of gameplay, both attacking and defending the stations.)

This was generally my experience when trying to balance stations as well. Fleet support-oriented stations like the midline style are typically both easier for the player to defend and simultaneously easier for the player to capitalize on its weaknesses, but it also is the most vulnerable station type as far as being surrounded and overwhelmed if not supported at all.

Making them both distinct and balanced is very tough, though. I've noticed certain statistics (drone regeneration rate as an example) have to be tuned to a very, very exact amount or they will just become overwhelming. Any deviance in those stats has to be handled carefully and requires fine tuning through multiple iterations of in-game trials with a specific fleet strength of the same ship variants. That doesn't even include weapon types as a factor. (Beams are less effective on high tech stations than low tech stations, etc, etc.)
Logged

FAX

  • Lieutenant
  • **
  • Posts: 53
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #12 on: October 31, 2019, 10:08:26 PM »

So exciting result, I could image your hard work for it!

About the experiment, will the final number be more accuracy by this?
  • A little bit autofit of station variant
  • Phase may not perform as well as heavy ships, the randomize of fleet composition may affect the final number(Too much Phase may leads to a higher one)
  • Increase shipsize 3 to 4,(size 3 means that the fleet include some frigates like lasher, which could do nothing but waste deploy points)

Emmm, My friends and I are interested in the raw data, are you willing to share it with me? I know this is a little unpolite, no matter how, thanks to do such a experiment.
Logged

Eji1700

  • Commander
  • ***
  • Posts: 221
    • View Profile
Re: Station Balance - Methodical Analysis
« Reply #13 on: October 31, 2019, 10:38:16 PM »

This all mostly tracks with my experience.

I don't think it's surprising though that the tier 3 midline is competitive.  Yes a player can exploit it if there's no fleet support, but as a station on your side, it's easily one of the best in vanilla.

The combination of slots it has tends to lead to a VERY well armed station that's exceptionally good at burst fire.  As long as you can deploy a fleet that can hold to the left and right of the station, the midline will erase 2-10 ships a rotation. 

The high tech is a general all rounder and I've felt that the only real issue the low tech has comes from it spending a ton of its mounts on PD, making it missile immune practically but it can struggle to actually kill since your AP damage can come mostly from slower weaponry (hellbores mainly)
Logged