I've been testing some things and I think I may be having a different issue. I also have been getting the mid and late-game crawl, but it seems to be related to the number of ships I have. Doesn't matter how many OP they have or if they're outfitted or not, but once I go over 10 I drop below 60 fps, and once I go beyond 15 I drop below 45 fps. Here are my computer specs and mod list:
The odd thing is that I spawned large fleets with this mod list and nuked them, probably about a dozen 500 point fleets, and I didn't get any slowness until I started scavenging ships.
Spoiler
Mods:
Lightshow
Trailer Moments
A New Level of Confidence
Adjusted Sector
Better Colonies
Beyond the Sector
Captain's Log
Caymon's Ship Pack
Combat Chatter
Console Commands
DIY Planets
Fluff Ship Pack
Hegemony Expeditionary Auxiliaries
Internal Affairs
Kadur Remnant
LazyLib
Legacy of Arkgneisis
Persean Chroniccles
Quality Captains
Stellar Networks
Supply Foraging
Tahland Shipworks
Vayra's Ship PAck
ZZ Audio Plus
Modified Industry Requirements
Graphicslib (disabled)
Spoiler
Specs
Windows 10
Processor Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz, 2592 Mhz, 6 Core(s), 12 Logical Processor(s)
Installed Physical Memory (RAM) 16.0 GB
Total Virtual Memory 29.3 GB
NVIDIA GeForce RTX 2060
I've allocated 4 GB of RAM to Starsector and never come close to that limit. I do seem to have large VRAM spikes, but typically float 1 or 2 GB free
EDIT:
Found the culprit in my case. It was Legacy of Arkgneisis, which is a shame because I quite like that mod.