Difficulty should comes from strategy and tactics, not campaign numbers and stats. With the new iteration of ECM, Officers, and DP distribution, battles are more one sided than ever. In the past we could grind fleets down with patience and a little skill no matter how hard, but now so much is stacked against a smaller fleet that its not even a contest.
Right now the current difficulty is 90% in "Do you have enough officers and ships? Do you have enough ECM? Can you cap the points before the enemy so you get to deploy?" If any one of those checkmarks is a no, the player loses. With the rare case of deploy a few ships and drag the AI into a corner to fight them little by little. This way is basically abusing the AI and should be fixed by the time the game goes final.
While interesting bosses are always welcome and puts a spin to the normal fleet fight and puts up a challenge, it should be the exception not the rule.
First problem is ECM. It is simply too powerful of a statistic. If it is in game, it should be special ECM ships that players and AI would have to protect. Like a shield-less Atlas on the field, where the AI and player will actively try to destroy. This way we turn a campaign suedo difficulty into something that could be interacted with, not just "I brought 10 officers and have skill checked."
Second problem is deployment. This whole, throw everything into a box and brawl it out is not going to be interesting with the wild different sized fleets running around. It fit well when the game was a battle simulator where both sides got equal points, but not anymore.
The fix?
The gold standard to strategy and tactic games is Final Fantasy Tactics. Not counting Econ RTS games like StarCraft etc... Tactics was a refinement of the strategy genera where the player is almost in every battle, fighting statistically superiors enemies. The game gives the player plenty of opportunity to use their wits and planned strategies to overcome powerful enemies. Everyone agreed the game was hard, but no one complained it was unfair, because AI had power, player had brains, and the devs allowed the player to use their brains.
BATTLE CHANNELS!!
Example:
In Tactics almost every battle had different areas players were forced to use if they wanted to deploy all their units, usually with minimum two areas, sometimes three or even with special events where the main character is alone surrounded. This gave many ways for the player to use tactics or strategy to overcome more and more greater adversity. If you have played the game and know what I am talking about, skip to the next one.
If not, imagine the enemy has twice the units/combatants, who has twice the HP of you. They have knights on the streets and archers on the rooftops, and a powerful mage behind their formation of knights. For the player to overcome such odds, they could deploy their mobile units like thieves and ninjas on the streets and jump everyone on the rooftops, fighting only half of the enemy forces at one time. Or, the player could have their defensive troops turtle on the ground, while having warriors on the roof tops killing the archers quickly and flanking the enemy knights and mage. As the game progressed and players got more options, the possibilities were expanded, while the difference between the player and the enemy widened even more.
In the end, it made the game more interesting. Allowed the player to fight statistically tougher opponents. The game was hard, but not unfair.
Integration:
This is merely a suggestion, but I hope it will give an idea of what could be possible.
So we make the battlefield larger, what shape exactly? Lets think using every shape we could think of while working this out. Also keep in mind sequential battles could also be a way.
Then, most importantly, we force both the AI and the Player to deploy all their ships. Hold your screams of "But the AI don't need Supplies and fuel!!!!"
Perhaps depending on engagement choices and AI fleet personality deployment areas are split into 2 or more areas where ships would have to travel a long distance to reach the other area.
Then, have the condition that when the AI fleet is not in an owned system, in addition to if all of their support ships are destroyed, they will surrender or flee (rational humans only ofc) no matter their advantage.
ECM like I said, could also be such a valuable target. Of course fleets could always not use ECM ships, but if such a powerful stat is in play, there should be counterplay.
With these changes we create multiple ways for players to encounter the battle. The player could sacrifice part of their fleet to concentrate on taking down the enemy support or warships. Does the player want to go in the capital and win the frontline battle, or does the player wish to play an assassin and lead frigates to destroy the enemy ECM or Supplies? What if the player encounter different factions or fleets that employ different tactics in splitting their ships differently?
This create the possibility of smaller fleets actually being able to do something to larger fleets. A small and aggressive ludic path groups? What will they do?
Right now we have mount and blade combat mechanics. With their cavalry, archers, and infantry units, they have a wider variety of types of units and how they function. So formation and tactics are important. But in Starsector right now, there is not early the same amount of depth to strategic or tactical side of combat. I think this is a great time to move away from this simple place-holder of a deployment system from battle simulator days and move on to something more complicated and interesting.