It can be due to a mod accidentally keeping object reference of combat engine so the Java is not able to garbage collect it, leaving a big chunk of memory occupied indefinitely.
The dead memory will add up each battle, slowly deteriorate the performance of the game and eventually force you to restart completely.
It would be helpful if you can give the list of mod you installed.
Sure, my currents mods as taken from my mod folder with their folder names is :
Spoiler
A New Level of Confidence - 40 - Rapid
Adjusted Sector v0.3.3
Another Portrait Pack
Arsenal Expansion 1.5.4a
Audio Plus
automatic-orders-0.3.2
Better Colonies
BeyondTheSector
BSC v0.1.2
CaptainsLog
Caymon's Ship Pack
Combat Chatter
Console Commands
DIABLEAVIONICS
enabled_mods.json
Exalted
EXPSP Kitbashes Only
Extratential Lanestate Union
FED
Grand.Colonies1.0.c
GraphicsLib
Hegemony Expeditionary Auxiliary
Hiigaran Descendants
HTE
Hyperdrive
Industrial.Evolution2.2.b
Kadur Remnant
LazyLib
Leading Pip
Legacy of Arkgneisis
Logistics Notifications
LowMaintenance
MagicLib
Mayasuran Navy
Missing ships
Modern Carrier
More HullMods
Nexerelin
Oculian Armada
Ocutek Pirates
OldHyperion
ORA
prv Starworks
Quality Captains
Roider Union
RotcesRats
SCY
SEEKER_UC
Shadowyards
SS-armaa-1.4.7
Starship Legends
tahlan-scalartech
tahlan-shipworks
Terraforming and Station Construction
Torchships
Transfer All Items
UnknownSkies
UpgradedRotaryWeapons
Vayra's Ship Pack
XhanEmpire