My point is there is no single metric and going that specific is pointless to misleading.
This method rewards versatility and the capability to speedrun relatively "weaker" enemies (compared to the maximum possible with the strongest/cheesiest fleets).
Other considerations can be how many stacks can a fleet do, how many strong loadout radiants can it handle,
how hard is it to lose something above a frigate if a mistake happens, recovery time between battles, ...
Perhaps, but it seems like the most straightforward metric: gauging the effectiveness of a ship (or a fleet more generally) based on how well it can absorb the enemy fleet's offensive power, while also doing offense of its own to kill the enemy fleet. Time to kill or the derivative damage per second are pretty standard metrics of this across countless games, and I explained above why this methodology captures both the offensive and defensive capabilities for fleets in Starsector.
Of course it measures via speedruns, since a faster run means either the fleet has greater offensive power (able to do more damage and kill more ships in the same amount of time) or the fleet has greater defensive power (able to neutralize more of the enemy's attacks, thus more of its flux goes toward offense), or both, and both are what a player would like to know for ship effectiveness when designing a fleet.
Your examples are either variations on DPS, or have nothing to do with combat:
* How many stacks a fleet can do essentially boils down to DPS multiplied by some function of PPT and CR decay, and is generally irrelevant beyond triple Ordos because the player will already be getting +500% XP bonus by then. It also favors infinite ammo weapons over limited ammo weapons (such as missiles).
* How many strong loadout Radiants the fleet can handle is the same as above, except now you're restricting it to a particular subset of ships. But a fleet of just Radiants never appear in vanilla, far as I'm aware, so you're just skewing the fleet composition toward anti-Radiants and away from actually dealing with Ordos fleets. Now if you have a particular enemy type that you want to fight (such as stations, or Derelicts, or whatever), you can use the same methodology but just using those fleets instead to fight against; I just find Ordos fleets to be the most useful to test against.
* How hard is it to lose ships is not only already included in my testing assumptions (any ship dying or having to retreat is automatic fail and don't count), but if a ship dies, it is no longer doing damage, so to maximize DPS you naturally want to keep the whole fleet alive.
* Recovery time between battles is a campaign-layer consideration and not a battle-layer one. Sure, the player might want to consider supply use, how much cargo it can hold, etc., when filling out a fleet, but that has no bearing on how well the AI uses these ships in battle, which is what we're discussing here.
Sure, if you squint and fudge a bit you can fit your data to the perceived value of some cruisers.
It's simply (total damage) / (battle time - 60), the 60 being how long I assume it takes for the two fleets to smack into each other (and which I may revisit in the future, but seems close enough for now). Anyone who understands rates such as miles per hour for speed or dollars per week for income should be able to grasp DPS as a measure of how quickly the enemy fleet is getting killed. There's no difficult math going on here.
As already mentioned it just doesn't work for specialized ships and undervalues tankiness/direct fire power (compared to overhead).
Monitor is tied with Gryphon for best ship in the game, worthless here. Bomber/squall Astral is very good, bad when spammed.
Dominator definitely worth more than ~21, spending 42 OP on Xyphos and trying to chase Lumens/Scintillas with it is just not a good idea. Falcon XIV at ~11.4 DP, it has very good alternatives but would be a complete steal there.
So let's take this as an example. How do you justify claiming that the Monitor is tied with Gryphon as the best ship in the game? What's your basis for saying this, beyond vague generalities like "because I feel like it is" or "it works really well"? How do you back up this statement in a way where some third party can look at your claims, reasoning, data, experiments, etc., and come to a similar conclusion?
For me it's pretty straightforward: after establishing the value of more generalized ships using the methodology, I start looking at specialized ships in terms of how they affect the overall
fleet, using the same methodology, just applied to a fleet of ships rather than spamming the same ship. For the Monitor, I'd look at (for example) how long it takes for 240 DP of ships to kill an enemy fleet, then (for example) how long it takes for 210 DP of those same ships + 30 DP of Monitors to take out the exact same enemy fleet. If the second fleet is faster, then I'd say those Monitors are more effective than their DP value (because presumably, they're absorbing so much of the enemy offensive power that the remaining 210 DP's worth of ships are able to do more damage faster than 240 DP of those same ships). I'd probably have to try out several different amounts of Monitors to figure out just what that right amount is.
But if you don't buy looking at how fast your fleet can kill an enemy fleet as a gauge of effectiveness, how do you justify the worth of a Monitor without reference to this? You don't advance the battle simply by sitting there tanking incoming damage; you advance the battle by having it let you accomplish more than you could otherwise.
(For what it's worth, I do feel like the Monitor is worth more than its DP; the question is how would you show this, or know in which situations is it the most effective. I also suspect that Alex deliberately under-costed its DP to encourage players to think more outside the box at fleet compositions rather than just all dakka all the time.)
~30-60 DP in Scarab/Glimmer/LP Brawler/Shrike/whatever else will capture what's needed and hunt down all the side frigates/destroyers much faster than any capital could.
It sure seems like you just implicitly assumed killing enemy ships faster as a metric for ship effectiveness, which is exactly my basis for using overall battle DPS.
It might not be the best method, but it is a classic method in the real world. "A is stronger!" "No, B is stronger!" Well, have them face off.
I think the main issue is that fundamentally it can lead to goofy results.
If we're basing A and B on the stock variants, then it just comes down to whatever default variants Alex put in, which has no particular reference to how well A would fare against B nor how well B would fare against A.
If we look for the best builds A and B that would do the best against each other, then you're basically going through iterations of finding the best build A that gives the highest win ratio against build B, then finding the best build B that gives the highest win ratio against A, etc. It's basically a competitive game in game theory. Due to the rock/paper/scissors nature of Starsector's combat, there's no guarantee that you'll ever converge on a stable solution for the best builds for A and B. And even if it converges, it might end up being due to some ridiculous build in the other ship that you would never see in the game, and thus wouldn't actually be a useful build to play with. For example a build that overly emphasizes PD because the other build's best way to win is to launch a bunch of Reapers.
But by testing them against a static target C, you can guarantee that you'll converge on a solution. Either the next build you try is better, in which case you keep it, or it's worse, in which case you discard it. You can try out different builds in this manner for A and B, and then see which one resulted in the highest score, however you define that to be. So it's a process that you know for sure you can get a good result.
I think comparing to the full skill setup and double Ordos and kill speed is interesting, but it is biased towards the particularly fit Remnant ships in those fleets (do the weapon fitting changes from fight to fight?).
It's the same double Ordos fleet that I use for all my testing. Basically after I gathered statistics on different Ordos fleets, I took 2 of them that were close to average in terms of size, fleet composition, etc., and have been using that same save to do all my double Ordos tests. So it's the same enemy fleet that the ships being tested fight against. I also have a different save for single Ordos fleet, and another one for triple Ordos fleets, again all selected to be close to average. However, the double Ordos save has seen the most use.
This statement got me thinking. If I remember correctly, you're using the Medusa flagship to distract and split up the Remnant fleet so that the bulk of your fleet can focus down a smaller portion at a time. Which also naturally reducing the incoming fire on the Conquests (or other ships under test). If so, is your human piloting potentially covering up a critical defensive weakness of Gryphons and Conquests, or other ships that are used in these tests, and thus biasing the results?
No, my Medusa is actually working to corral the enemy fleet together, not spread them out. It does by necessity distract the enemy fleet when it goes in to attack (because a destroyer close by is much more attractive to most ships than a capital ship far away) but splitting up the enemy fleet would actually make my fleet fare worse; I want to surround the enemy fleet, not the other way around, and splitting them up would help them surround my fleet.
Rather, my fleet can focus on a smaller portion of the enemy fleet simply by killing them fast enough that most of the enemy fleet is still moving toward the front lines rather than actively firing at my ships. The beginning of the battle is the most dangerous part because I only have 200 DP to start with against their 240 DP, so I have some pressure to kill off enemy ships quickly. Once that happens though and my fleet is in a U-shaped semi-circle around the enemy fleet, then it's just gradually moving upward toward their spawn point, killing them in a stream. This is basically the Starsector version of
crossing the T; at some point I worked out once this happens, my 240-DP fleet is basically just facing an 80-DP fleet at any given time, until the end when the Radiants show up. But at that point the Radiants are surrounded by ships so they can't really mount any sort of proper offensive since they're taking fire from all sides, and they're finished off quickly.
I posted a video before of the flagship Medusa using SO in a Conquest fleet and running around killing stuff. It turns out though that it was counterproductive; whenever I rushed in to attack, I would take damage from the Conquests trying to fire at the enemy ships, plus then they would stop firing because I'm in the way. It turns out that it's better for me to just stay back and let them do their work. Thus I switched over to a non-SO build so I could stay farther away from the targets (and switched out of Cryoblasters so I could assume no Omega weapons), stayed more next to the Conquests, and the results were much better; the result I posted above was finishing in 316 seconds, which was over a minute faster than the 379 seconds when I was using the SO Medusa. So it was better to just let the Conquests do their thing than to try to get into the thick of it.
If that's the case it's pretty skewed, an easy way to cheese most fleets is to park the main force and keep killing the small groups sent to capture points with small groups of fast ships.
Main fleet never sees a full engagement, and doesn't have to deal with the Radiants spawning all at once at the end of the battle.
That's a
terrible way to maximize DPS. If you're looking to kill enemy ships quickly, your ship should be firing on them as often as possible, as much as your flux allows. That means having another enemy ship nearby waiting to get fired upon as soon as you're done killing an enemy ship. Waiting for the enemy fleet to trickle in a frigate or two at a time to each objective as they arrive and get killed means your fleet is spending most of its idling.
Instead I have my fleet fan out into a line across the map at the beginning, enough to encompass the objectives. Once they kill off the initial fleet and have captured the objectives, they then gradually move toward the enemy spawn point; usually at some point I set them to full assault. After that they just do their own thing, except if I need to order them to do something like chase down a frigate that's wandering away or something. That's the fastest way I've found to kill the enemy fleet, and the strategy I use for testing fleets.
