Fractal Softworks Forum

Please login or register.

Login with username, password and session length
Pages: 1 [2] 3 4 ... 9

Author Topic: 0.9.1a Balance Testing Case Study: Condor  (Read 12505 times)

Thaago

  • Global Moderator
  • Admiral
  • *****
  • Posts: 7214
  • Harpoon Affectionado
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #15 on: July 10, 2020, 09:39:49 AM »

These tests are very very interesting! I'm shocked that the Condors are winning. I would not expect that at all, especially with them not even using their medium missile mount, but I guess quantity of fighters means a lot/reserve deployment is less powerful in AI hands than I'd assumed. I would normally advocate using cheaper fighters on Condors to get that mount filled, or ditching the point defense to do the same, but I can't really argue with results.

[Edit]
I'd also love to see the results of tests where the Condors have Salamander Pods, because of how great they are with fast missile racks.
« Last Edit: July 10, 2020, 09:43:56 AM by Thaago »
Logged

Mondaymonkey

  • Admiral
  • *****
  • Posts: 777
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #16 on: July 10, 2020, 09:54:38 AM »

So... Condors ain't that bad ah?

I wonder if 60 DP amount of condors would do with Paragon?  :o

Logged
I dislike human beings... or I just do not know how to cook them well.

Hiruma Kai

  • Admiral
  • *****
  • Posts: 886
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #17 on: July 10, 2020, 10:10:14 AM »

So, switching to Talons had the Drovers perform much better.  Although, the Condors still won a fight, technically.  More like a draw to be honest.

Being able to fully leverage Reserve Deployment apparently matters.  One might argue put interceptors on Drovers, and bombers on Condors.  To be honest, both Drovers and Condors generally are able to stay out of the way with a wall of gunships in front, so the 40 vs 70 speed at the end of the day doesn't seem to matter much in these kinds of engagements.  Fighters are just so long range.

Condor
2x Talon, Harpoon pod, 2x Vulcan, Expanded Deck crew, rest in caps

Drover
2x Talon, 4x Harpoon, 2x Vulcan, Expanded Deck crew, rest in caps/vents

Drover victory, Lost Wolf, Lasher, Hammerhead.
Drover victory, lost 2x Wolf, 2x Lasher
Condor victory, lost 2x Wolf, 2x Lasher, 2x  Hammerhead, 6x Condor (i.e. Eagle was only ship on the field at the end, and at 0% CR)
« Last Edit: July 10, 2020, 10:12:25 AM by Hiruma Kai »
Logged

Yunru

  • Admiral
  • *****
  • Posts: 1560
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #18 on: July 10, 2020, 11:04:24 AM »

I ran the Onslaught test myself a dozen times because I'm certain the AI is symmetrical. From what I saw, the battles usually follow one of four scenarios:
Both Onslaughts use their system immediately and the battle is even.
Both Onslaughts use their system late, bump into each other, often flameout, then the battle is more or less even depending on the flameouts.
Either Onslaught uses their system after a few seconds, wins the flux war because the other Onslaught has its shield up, and ends up winning with a huge margin. For some reason, the "player side" Onslaught did that a lot more often than the other, but both versions of this scenario did occur.

Therefore I'm confident to say that the mission does NOT affect the AI in any way and it is working as intended.
My thanks, and also my damnation, for now I must find a more consistent metric to re-re-price my ships XD

pairedeciseaux

  • Captain
  • ****
  • Posts: 340
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #19 on: July 10, 2020, 02:45:21 PM »

Thank you very much Hiruma Kai, for running all those tests and also sharing both results and meaningful analysis. You did an awesome work!  :)

A few comments below about the first test. Wall of text warning.

Here we have a small fleet, looking like one a player would have during the first campaign hours. This "player" fleet encounters what looks like a 40K pirate bounty fleet. Not exactly like a real pirate fleet, because here we have pristine ships, and a Lasher gets added to spice things up. For any experienced player, in a real campaign there is not much risks to loose the fight or even loose a ship. So what value does the test bring, if any?

Let's say you are the player, currently wondering whether to add a ship to your fleet, or not. You are currently at a market, knowing what your next encouters will be. You have a mix of cash, weapons, and maybe fighters. Real campaign situation, right?

Not all players are experienced enough to manage the fight itself nor make an informed decision about what to do with the cash beforehand. Also keep in mind the test itself is AI vs AI: no advanced player tactics, no player priorization, no player fleet coordination, no carrier Strike order, no Defend order, no Escort order, no retreat order. You get the idea, this is a gameplay-less battle simulation.

Here are your options:
  • buy and outfit a Condor
  • buy and outfit a Drover
  • buy and outfit a Hammerhead
  • keep your cash to buy later something you can't buy now

Obviously there are more options in real campaigns (more frigates!), but let's keep the test simple.

What do you choose? There are no good or bad answers here, IMO, all are valid early game options.

Let's have look at the results from Hiruma Kai's first test report, especially battle duration. I took the liberty to separate them in two groups.

First group with early game fighters (Talons) and no carrier options:
  • with Condor + Harpoon Pod + 2 Talon wings: around 2min40 and 100% win, unstable results
  • with Drover + 4 Harpoons + 2 Talon wings: around 2min40 and 100% win, stable results
  • with Hammerhead: around 3min20 and 100% win, unstable results
  • no additional ship: around 4min00 and 100% win, stable results

And second group with more advanced fighters - Kopeshs and Daggers - mixed with Broadsword:
  • with Condor + Pilum + 1 Broadsword wing + 1 Kopesh wing : around 4min10 and casualties, unstable results
  • with Condor + single Reaper + 1 Broadsword wing + 1 Dagger wing : around 3min10 with 100% win, unstable results
  • with Drover + 4 Harpoons + 1 Broadsword wing + 1 Kopesh wing : around 3min00 and 100% win, stable results
  • with Drover + 4 Harpoons + 1 Broadsword wing + 1 Dagger wing : around 3min20 and casualties, stable results

Notes about using those advanced fighters: IIRC Kopeshs and Daggers (also Cobra) cost a lot, are harder to find, and those with unguided missiles will underperform against frigates. And... maybe Atropos got caught by point defence and tanked by shield?

Other than that, looking at the first group there is a clear hiearchy:
  • both Condor and Drover are at the top: adding 1 carrier with 2 Talon fighters brings best value to your fleet againsts frigates compared to any other options seen here
  • next is Hammerhead: unsurprisingly sluggish against frigates compared to fast fighters, 40 additional seconds to win compared to carriers
  • the no additionnal ship option is last: with an awful time to win, the time and supply cost necessary to recover from this fight could be a real problem in real campaign

One more observation: with Drover we have consistent stable results (time to win), with Condor we don't. It could mean that overall Drover is more reliable compared to Condor. It would be hasty to conclude that way with only one test scenario, but at the same time it mirrors some player's experience. So basicilly let us say that, if given a choice between Condor and Drover (cash and ship availability conditions being met), player choose between lower-price-lower-reliability and higher-price-higher-reliability. And very early game, as long as player don't rush fighting dangerous fleets, both solutions will perform equally well.

Caveats: one should be wary of putting too much faith in statistics, especially with a lowish number of repetitions for each run. Also real player test run should always be used as a control result, here I can say the results Hiruma Kai obtained with both Condor option and no additional ship option match my real campaign experience from a few days ago.

One very last thing I'd like to mention, despite being outside the scope of the test as I imagined it initially, I feel like fighter wings choice might prove as important or more important than the choice of the carrier itself. Some of the results are showing this, but I suspect it would make our job much harder to evaluate the effect of those 2 variables at the same time.

[edit: added missing casualties in the test results, removed the "no casualties" mention in the "One more observation" paragraph]
« Last Edit: July 10, 2020, 04:07:15 PM by pairedeciseaux »
Logged

Hiruma Kai

  • Admiral
  • *****
  • Posts: 886
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #20 on: July 10, 2020, 03:31:29 PM »

First group with early game fighters (Talons) and no carrier options:
  • with Condor + Harpoon Pod + 2 Talon wings: around 2min40 and 100% win, unstable results
  • with Drover + 4 Harpoons + 2 Talon wings: around 2min40 and 100% win, stable results
  • with Hammerhead: around 3min20 and 100% win, unstable results
  • no additional ship: around 4min00 and 100% win, stable results

And second group with more advanced fighters - Kopeshs and Daggers - mixed with Broadsword:
  • with Condor + Pilum + 1 Broadsword wing + 1 Kopesh wing : around 4min10 and casualties, unstable results
  • with Condor + single Reaper + 1 Broadsword wing + 1 Dagger wing : around 3min10 with 100% win, unstable results
  • with Drover + 4 Harpoons + 1 Broadsword wing + 1 Kopesh wing : around 3min00 and 100% win, stable results
  • with Drover + 4 Harpoons + 1 Broadsword wing + 1 Dagger wing : around 3min20 and 100% win, stable results

It might be due to my poor formating and so you may have missed it, but I'll note the Drover + 4 Harpoons + Broadsword + Dagger had a Wolf casualty on its first run, same as the Condor + Pilum + Broadsword + Khopesh did on its 2nd, so to be fair it should probably be listed there.

Having watch some of the fights (and left it to run in the background on others), whether the wolf dies or not is more due to the AI and the intial exchange than the fleet composition.  Just sometimes it misjudges badly.  I mean, the smallest fleet had no losses, while each of the carrier fleets lost 1 wolf once, and the bonus Hammerhead almost lost one.  Pretty sure at this concentration of force, a Wolf can simply race ahead, teleport in, fires it AM Blaster, and then simply eat too many missiles before it can back off enough.  Bad timing of firing a huge flux costing weapon. A PD wolf or other less flux intensive variant probably would have been fine.

I also would be extremely hesitant that 3 runs imply a particular stability in terms of speed of completion.  The sample size is too small, and depends on how the AI ended up dividing into smaller skirmishes.  The fastest run, at 1 min 50 seconds was a perfect storm of the Buffalo going pop in the very first exchange of missiles, like at 45 seconds in, at which point without the enemy destroyer, the other ships moved up confidently, and the talons chased down the fast frigates quickly.  They were all together which made it quick.  That could have happened with any of the fleet compositions, but the Condor fleet got lucky with that. 

However, in other fights, you've got a pair of shepherds facing off in the bottom right while everyone else is fighting in the upper left, and you've got like 30-45 seconds of travel time until a wolf finally gets over there to finish it.

Keep in mind, this also all without human intervention.  A human player can be much more aggressive (presumably piloting the Hammerhead), and possibly requesting coordinated fighter strikes or escort with command points, which will tend to make things faster and safer in this kind of setup.

I considered doing the piranna test, but those bombs are so slow and this was a fairly agile enemy fleet, that I felt they wouldn't contribute based on previous experience, so didn't feel like taking 30 minutes to test.  Generally when I'm playing, even at the start of the game, if I see a higher end fighter longbow or dagger, I generally prioritize a purchase just to have a minimum of 2 decent fighters by the time I move into carriers.

Personally, I think in terms of comparisons, the larger 60 DP gunship + 60 DP carrier tests are more telling than the time to kill, as they're much more sensitive to a slight imbalance because they snowball.  That yes/no helps provide a definite signal - but unfortunately doesn't tell us how much, just that it is due to the snowball nature.  I'm pretty sure a human just giving orders, not even piloting, could shift victory either way very easily.

To throw one more data point on, I did a 120 DP gunship + 120 DP carrier fleet (so 240 DP total for each side), broadsword + dagger drovers vs condors just for kicks.  Long fight, but condor still pulled it out.  Lost an Eagle, hammerhead, Condor, 3x Wolf in process, but those 2 extra carriers definitely help as things scale up.
Logged

pairedeciseaux

  • Captain
  • ****
  • Posts: 340
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #21 on: July 10, 2020, 04:19:17 PM »

It might be due to my poor formating and so you may have missed it, but I'll note the Drover + 4 Harpoons + Broadsword + Dagger had a Wolf casualty on its first run, same as the Condor + Pilum + Broadsword + Khopesh did on its 2nd, so to be fair it should probably be listed there.

Good catch! I just fixed it in my previous message. This is the kind of thing that should ideally be generated rather than hand-edited to avoid silly mistake like I made (100% my fault, by the way).

Having watch some of the fights (and left it to run in the background on others), (...)

I also would be extremely hesitant that 3 runs imply a particular stability in terms of speed of completion.  The sample size is too small, and depends on how the AI ended up dividing into smaller skirmishes.

Yes, if possible we want some automation and/or speed increase to perform more runs. And recruit more people to execute test campaigns.

To throw one more data point on, I did a 120 DP gunship + 120 DP carrier fleet (so 240 DP total for each side), broadsword + dagger drovers vs condors just for kicks.  Long fight, but condor still pulled it out.  Lost an Eagle, hammerhead, Condor, 3x Wolf in process, but those 2 extra carriers definitely help as things scale up.

Ah!
Logged

pairedeciseaux

  • Captain
  • ****
  • Posts: 340
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #22 on: July 11, 2020, 02:50:55 AM »

Personally, I think in terms of comparisons, the larger 60 DP gunship + 60 DP carrier tests are more telling than the time to kill, as they're much more sensitive to a slight imbalance because they snowball.  That yes/no helps provide a definite signal - but unfortunately doesn't tell us how much, just that it is due to the snowball nature.  I'm pretty sure a human just giving orders, not even piloting, could shift victory either way very easily.

Good points.

I'm no pro statistician, and I don't have deep knowledge of Starsector's inner workings. That said, IMHO what matter when designing a test is asking one self some key questions:
  • What insight do I seek?
  • What metric(s) would best provide such insight?
  • How do I gather metrics and build insight without introducing bias?
  • What is the appropriate sample size?
  • What are the caveats? Any identified bias and/or limitations?
  • (duh, anything else?)
  • How do I report the insight?

About metrics, in some case discrete value sampling is the straighforward and best approach, such as:
  • won battle / lost battle
  • ship X died / ship X survived
  • killed 0 cruiser / killed one cruiser / killed two cruisers

And continuous value can also be helpful, such as:
  • battle duration
  • total damage inflicted by ship X
  • total damage sustained by ship X
Logged

Grievous69

  • Admiral
  • *****
  • Posts: 2991
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #23 on: July 11, 2020, 03:02:25 AM »

So what I've learned from all of this is that fighter superiority is even more important that I thought originally. I knew that it's good to have a carrier or two in your fleet just to help with those pesky frigates or just pure distraction, but this, this is just wrong. I wonder how a fleet full of Converted Hangars would perform vs just the ''usual builds'' (I know it's not the thread for this, I'm just thinking out loud). So yeah long story short, fighters are too strong and AI is too dumb vs fighters.
Logged
Please don't take me too seriously.

pairedeciseaux

  • Captain
  • ****
  • Posts: 340
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #24 on: July 11, 2020, 04:14:14 AM »

On another note, Hiruma Kai, I followed the suggestion you made in the other thread, and sent a PM query to our lord and saviour our testing master Dark.Revenant. He shared some precious informations and advice in his reply. For anyone looking to perform more testing of this kind, the summary is (hopefully I didn't distort it too much):
Spoiler

Dark.Revenant's test:
  • https://fractalsoftworks.com/forum/index.php?topic=15758
  • in his Interstellar Imperium mod there is a mission that helped provide a testing framework : Station Tester, he used the mission and manual operation to collect data
  • I just did a quick check: looking at the various mission file in the mod I see "ii_test1"="Station Tester", "ii_test2"="Fleet Tester", "ii_test"="Ship Playground"
  • currently there is no automation to run a series of battles and collect results, but it would be possible to create such a framework

So in order to increase the number of repetitions, my understanding of what testers could do:
  • use the mission provided by Dark.Revenant in II mod, it basically allows to obtain 3x battle simulation speed
  • run several parallel instances of Starsector on a PC/workstation, it further increases the overall speed (depending on the number of instances the hardware can handle), suggested by Dark.Revenant
  • have several people contribute to the same test, then merge results, a strange idea of my own that in one hand pose some coordination challenge and additional mistake risks, but on the other hand provide additional test controls (having each tester obtain similar results helps build confidence in the metrics)

By the way, at this point I haven't attempted to run any test, I'm still relatively clueless on the test execution part, and have spent too much time reading and writing walls of text on the forum.  :D
[close]
Logged

Hiruma Kai

  • Admiral
  • *****
  • Posts: 886
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #25 on: July 11, 2020, 05:50:18 AM »

So what I've learned from all of this is that fighter superiority is even more important that I thought originally. I knew that it's good to have a carrier or two in your fleet just to help with those pesky frigates or just pure distraction, but this, this is just wrong. I wonder how a fleet full of Converted Hangars would perform vs just the ''usual builds'' (I know it's not the thread for this, I'm just thinking out loud). So yeah long story short, fighters are too strong and AI is too dumb vs fighters.

Well, I wouldn't say it is superiority.  Its more like, concentration on target.

Here's some more data to throw out there.

I only did one run of each of the following so far.

60 DP gunship core + 6x Condor
Condor with Broadsword, Dagger, Reaper, 2x Vulcan, Expanded Deck crew, rest in caps

vs

60 DP gunship core + 3x Heron (Cruiser class carrier)
Broadsword, 2x Dagger, Expanded Deck crew, x3 PD Laser, x1 Flak, 20 Caps, 1 vent

Very clean Heron victory, lost Wolf, Lasher, but it did not go to time unlike nearly all the other fights.  Coordinated bombers were taking out ships in like 1 to 2 passes.
Heron also seems to have much better timing on using its ability.

In this case, the Condors have 6x Broadsword and 6x Daggers, but are losing to 3x Broadsword + 6x Daggers (admittedly with the ability to burst damage by x1.5).  But are very clearly taken out.  Also, unlike a Drover sending their fighters against a Condor, the Condor doesn't survive.  Condor can tank 3x Atropos at a time roughly indefinitely.  Not so much against 6x1.5=9.

I then tried to have the same ratio of broadsword to daggers on the condor side by using 2 different condor designs.
Condor with Broadsword x2, Salamander pod, vulcan x2, expanded deck crew, rest in caps (2 of these)
Condor with Broadsword, Dagger, reaper, vulcan x2, expanded deck crew, rest in caps (4 of these)

versus the above Heron fleet.

Heron still win, no losses this time.  Again a very quick fight.

Once a coordinated group of bombers is large enough, and can deal enough damage to overload a target in a single pass, you hit a transition point.  Things start dying quick.  Hammerheads, Condors and Frigates just can't deal well with the damage equivalent of 9 (or sometimes 18) 1000 damage HE missiles in a short period of time.  Especially if they're already in engaged.  The coordination (all of them hitting in a short period of time) really makes the difference.

So out of curiosity, I then decided to try the Herons against the Drovers.
Drover with Broadsword, Dagger, 2x Vulcan, 4x Harpoon, Expanded Deck crew, rest in caps

Heron won, 2x Wolf, 2x Lasher, 1x Hammerhead

But it was a long fight.  The frigates ran out of CR.  The superior shields and speed of the Drovers begin to tell in the case where the enemy is superior.  They survive longer by kiting.  It doesn't make them win, but it does in fact mean they survive longer when your side is losing.  Which can be extremely important for a player fleet, given a human pilot can be superior locally where they're flying, but have their fleet be inferior everywhere else. 

Which suggests we should also do some testing of something like 240 DP fleets against 120 DP fleets or the like.  Put the carriers on the losing side, and see how long they last, and how many kills they get.  That should show the Drover superiority.

In our previous testing, the speed and toughness of the Condor just needs to be high enough to survive a pair of fighter wings, as that's the only thing that can reach them.  Because the AI doesn't focus fire, as long as the Condor can survive that shield tanking, it'll survive indefinitely.  Since the offensive power of 5 Drovers is close to or slightly lower to that of 6 Condors (reserve deployment only really worth 1.5 on fighters/interceptors, and even then inefficiently on cooldown), we don't see a big advantage.  Mostly because we're not stressing that part of the ship stats.

Also other interesting AI tidbits.  The Herons in all cases were much closer to the front lines.  Being cruisers, they seemed to fear the other ships less.  Which meant shorter turn around times for fighters, and perhaps more importantly, they tanked some bomber waves, and then would pull back to vent.  Which meant the fleet had effectively more shield capacity to spread incoming damage across at the beginning of the fight.

It'd be interesting to see what happens with the drovers if we put railguns instead of vulcans on, and bump their aggressiveness up.  Potentially if you use them more like 2nd string brawlers + fighters, they might do better compared to Condors, as you bring their flux stats into play in the beginning, spreading damage out, and giving the other ships more breathing room.  Or they might fly in front of their allies and get blown up while blocking shots.  Would definitely need testing.
« Last Edit: July 11, 2020, 05:58:30 AM by Hiruma Kai »
Logged

SCC

  • Admiral
  • *****
  • Posts: 4142
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #26 on: July 11, 2020, 06:21:10 AM »

I tested using of Fleet Tester that included player-side admiral AI, so test results might be different than yours. However, the upside is that the testing is completely divorced from any skill in playing Starsector I have (except for making loadouts). It also meant that some ships were retreated off the battlefield, at discretion of admiral AI.

Player fleet is composed of 60 DP carrier formation, 1 Conquest (Elite), 1 Eagle (Assault), 1 Venture (Balanced), 2 Enforcers (Elite), 1 Hammerhead (Balanced), 2 Centurions (Assault) and 2 Wolfs (Assault). 60 DP carrier formation stood for one of the four: 6 Condors, 5 Drovers, 3 Herons or 3 Moras. Non-carrier ships are all vanilla, unchanged variants, except for Venture, where I swapped Mining Blaster for Pulse Laser, because I hate it that much.

Enemy fleet consists of 1 Onslaught (Standard), 1 Eagle (Balanced), 1 Falcon (Attack), 1 Heron (Strike), 1 Mora (Strike), 1 Condor (Support), 1 Drover (Starting), 1 Medusa (Attack), 1 Sunder (Close Support), 2 Shrikes ((P) Attack), 1 Wolf (Strike), 1 Tempest (Attack), 1 Omen (Point Defence) and 1 Lasher (Assault). These are all unchanged vanilla variants. This fleet had a slight DP edge over the player fleet.

Condors either came as all strike variants or half strike, half support. I switched between these compositions on a whim and I don't remember anymore when which set fought, but it's close to a half. Strike Condors had 1 Broadsword wing, 1 Perdition wing, 1 Harpoon Pod, 1 Vulcan and Expanded Deck Crew. Support Condors had 2 Thunder wings, 1 Harpoon Pod, 2 Vulcans, Expanded Deck Crew and the rest spent on caps.
      In 12 tests, player fleet with Condors won 1 time. They often fell prey to enemy flankers.

Drovers were most often fielded in 3:2 ratio of assault and strike Drovers, though once I made a mistake and run all strike Drovers. Assault Drovers had 1 Talon wing, 1 Broadsword wing, 4 Harpoon Racks, 2 Vulcans, Expanded Deck Crew, Expanded Missile Racks, 8 caps and 8 vents. Strike Drovers had 1 Broadsword Wing, 1 Dagger wing, 4 Harpoon Racks, 2 Vulcans, Expanded Deck Crew and Expanded Missile Racks.
      In 11 tests, player fleet with Drovers won 7 times. Drovers seemed to die mostly after they run out of PPT and were vulnerable.

All Herons had the same strike variant. 1 Broadsword wing, 1 Dagger wing, 1 Khopesh wing, 3 LRPD lasers, 1 HVD, Expanded Deck Crew, Dedicated Targeting Core.
      In 5 tests, player fleet with Herons won 3 times. Similarly to Drovers, they died mostly after running out of PPT.

Moras also came only in one variation. 1 Broadsword Wing, 2 Perdition wings, 2 Typhoons, 2 Railguns, 5 Vulcans, Expanded Deck Crew, rest in vents.
      In 3 tests, player fleet with Moras won 2 times.

Grievous69

  • Admiral
  • *****
  • Posts: 2991
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #27 on: July 11, 2020, 06:32:39 AM »

Yeah that seems about right. Once the fleet with Condors has actual pressure put vs them, it collapses faster than other setups. Which is precisely why I stop using them in campaign after tutorial or so, since I'm usually fighting bigger fleets than my own. So in the end it seems it has a use in campaign. If you already don't have carriers and can't find anything better (or can't afford), a Condor will improve your fighting capability by a significant margin. But after you earn some cash and have better options, there's little reason to keep them. I guess they aren't THAT bad as I was claiming, I wouldn't mind a small buff still tho.
Logged
Please don't take me too seriously.

pairedeciseaux

  • Captain
  • ****
  • Posts: 340
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #28 on: July 11, 2020, 07:26:49 AM »

Well done, SCC!

Two comments on your test runs.

(1) You fighter wings composition changes across your player carrier formations, while Hiruma Kai's runs stayed consistent. So your tests have a heavy bias on fighters wing composition.

(2) Indeed this is the hard core test for Condor. Having so many hunter killer ship in the enemy fleet that can catch the slow Condors. I would say, given the ratio of number of carriers / 10 non-carrier ships, that situation is "unfairly" emphasised the in Condor case. Because the Condors have a higher chance of being targeted, and when they are, won't receive much help with this fleet composition.

Comparing this to a real campaign situation: you would realistically have a lower number of Condor (those left from early game), and more non-carrier ships, and probably dedicated escorts. This what I have in my current campaign: 2 Condors, 3 Herons, 8 frigates, 2 destroyers, 3 cruisers, 1 Legion recently added. Condors are not being targeted that much, and have a permanent 1 Shepherd escort. I have left them unmanaged since early game with 100% survival rate.

That doesn't mean your test scenario is irrelevant, quite the contrary. Just this, IMO: caveats mentioned above, YMMV, and so on.
Logged

Hiruma Kai

  • Admiral
  • *****
  • Posts: 886
    • View Profile
Re: 0.9.1a Balance Testing Case Study: Condor
« Reply #29 on: July 11, 2020, 07:42:35 AM »

I tested using of Fleet Tester that included player-side admiral AI, so test results might be different than yours. However, the upside is that the testing is completely divorced from any skill in playing Starsector I have (except for making loadouts). It also meant that some ships were retreated off the battlefield, at discretion of admiral AI.

Thanks very much for the information.  I don't suppose you know if the Admiral AI you are using the same one used in the AI battles mods?  I used it assuming it'd been setup with at least some form of fleet AI for both sides.  Alternatively, I'd be interesting to know which mod to grab that has said Admiral AI.

Definitely interesting data points.  Where all individual ships set to steady AI?

Although I'm wondering how much its affected by fighter selection versus base ship.  I'd like to try this fleet setup, but with all 4 carrier types using the same ratios and types of fighters, to help eliminate other variations beyond the base ship.  Sounds like the flanking ability of the enemy fleet was superior compared to the player fleet seeing it had Medusa, 2x Shrike, Wolf, Tempest, Lasher, Omen.  Fast ships on the carrier side seems to be 2x Centurions and 2x Wolves, plus presumably the fighters.  In my tests, the flankers tended to get paired off since it was much more a mirror match.

So far I've run 1 match with the AI battles mod, using your setup, except using just broadsword/dagger/reaper/vulcan x2/expanded crew deck condors instead of Broadsword/Peridition or Thunder wings.  So far, 1 for 1 victory for the condors. I'm going to do a few more of those (to see if the Condors just got lucky), as well as Condor with broadsword/peridition + 2x Thunders and see how it does for baseline comparison with identical testing mod.  Might reveal AI is doing a better job of ordering coordinated fighter strikes or something.  And then I'd like to see how the Drovers do with Broad/Peridition + 2x Thunder.
Logged
Pages: 1 [2] 3 4 ... 9