Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[erikem]

Hi,

I play a game with Nexelerin and bunch of factions, big galaxy (or is it called standard?). Lotsa factions but only a few select have more than 2 markets. My problem is that right now my compressed save file is already 61mb and save/load process takes uncomfortable amount of time even on SSD.

Is it expected? Is there any way to trim down the save file size or make saving/loading faster?

[SCC]

Yes. Also yes, play with fewer mods. Mind you, 0.9 already saw major loading optimisation, so it could have been worse. You just have to live with it.

[erikem]

And if my make "small" sector? Will it help or is it going to be the same in the long run?

Also how does "having less mods" actually help? Isn't the size of the save file driven by the number of active markets and fleets? I mean if I have 10 factions x2 markets each, x10 fleets each (20 markets and 100 fleets in total) will it be different from 2 factions with 5 markets and 50 fleets each?

[Alex]

Hmm, this kind of makes me wonder whether mods are taking advantage of the "routemanager" system for inflating/deflating fleets and so on. 60mb seems like surprisingly a lot; I'm not sure what would take up that much space. Something seems off about that.

My problem is that right now my compressed save file is already 61mb

Just to confirm, you've got savefile compression turned on via settings.json? It might actually be faster with that turned off, especially if you've got an SSD, since disk writes are cheap and compressing a large savefile is less so.

[erikem]

Just to confirm, you've got savefile compression turned on via settings.json? It might actually be faster with that turned off, especially if you've got an SSD, since disk writes are cheap and compressing a large savefile is less so.

Yup, save file compression is turned on via settings.json:
"compressSaveGameData":true

Uncompressed campaign.xml is like 277mb. Actually the speed seems to be the same both with and without compression and probably that's coz of otherwise fast PC.

I also have kind of a vague feeling that save file went almost twice as large when I started exploring the unknown but that could simply be a coincidence.

I do "save copy" quite often and here is the dynamics (please disregard the top save with 399mb size; compression was enabled since save_ErikemII_7906960025789190898; some of the saves might look suspiciously small because I never did the second save on that file (did "copy save" and next time did "copy save" again) and thus they don't have *.baks):

[Midnight Kitsune]

Holy S***! I've NEVER seen save files that big! I THINK I've heard of a mod causing save bloat issues (Vayra's sector I THINK). But yeah, I'm pretty sure the shear sizes of those saves is what is taking you so long to save your game

[Alex]

... oh, wow! Yeah, this is definitely not normal-looking; as Midnight says, this looks like it's likely a save bloat bug in one of the mods you're using.

[erikem]

Just to be clear these sizes are from the saves before I turned on the save file compression. Here are the save after I turned on the compression:


Also some of the saves might look suspiciously small because I never did the second save on that file (did "copy save" and next time did "copy save" again) and thus they don't have *.baks

Holy S***! I've NEVER seen save files that big! I THINK I've heard of a mod causing save bloat issues (Vayra's sector I THINK). But yeah, I'm pretty sure the shear sizes of those saves is what is taking you so long to save your game

Yup... I kind of understand that the problem is in the sheer size =( And yes I do have "Vayra's sector". Are there any other mods infamous for save bloating?

Here's the list of mods I've been using:

Spoiler

{"enabledMods": [
  "$$$_trailermoments",
  "anotherportraitpack",
  "raccoonarms",
  "blackrock_driveyards",
  "CombatAnalytics",
  "chatter",
  "lw_radar",
  "istl_dam",
  "diableavionics",
  "DisassembleReassemble",
  "gs",
  "GMDA",
  "HMI",
  "HA",
  "junk_pirates_release",
  "lw_lazylib",
  "leadingPip",
  "ArkLeg",
  "luddenhance",
  "MagicLib",
  "Mayasuran Navy",
  "nexerelin",
  "ORA",
  "sun_ruthless_sector",
  "sad",
  "SCY",
  "shadow_ships",
  "swp",
  "bonomel_skilledup",
  "speedUp",
  "Sylphon_RnD",
  "tahlan",
  "underworld",
  "US",
  "URW",
  "vayrasector",
  "lw_version_checker",
  "audio_plus",
  "shaderLib"
]}

[close]

[erikem]

... oh, wow! Yeah, this is definitely not normal-looking; as Midnight says, this looks like it's likely a save bloat bug in one of the mods you're using.

Any ideas how can I identify the mods causing save bloats? Apart from playtesting various mod combinations which could take like eternity =)

[Alex]

Any ideas how can I identify the mods causing save bloats? Apart from playtesting various mod combinations which could take like eternity =)

Well, you could try to open up the xml in some kind of editor that can handle files this big, and see if anything jumps out as taking up a lot of the file, but that could be pretty tricky to figure out.

Beyond that, yeah, testing with combinations of mods would be the way to go. Probably fastest to do half the mods (aside from the utility mods) at a time, so that you can tell which half of the mods the problem is in, and then progressively narrow it down from there. Still quite a bit of time/effort, though.

... I THINK I've heard of a mod causing save bloat issues (Vayra's sector I THINK). ...

And that might be a good one to start with as just a one-off...

[erikem]

Well, you could try to open up the xml in some kind of editor that can handle files this big, and see if anything jumps out as taking up a lot of the file, but that could be pretty tricky to figure out.

Notepad++ can handle it but I'm not sure I'll be able to find the issue. It's not like each mode is putting it's own stuff in a separate big block

Beyond that, yeah, testing with combinations of mods would be the way to go. Probably fastest to do half the mods (aside from the utility mods) at a time, so that you can tell which half of the mods the problem is in, and then progressively narrow it down from there. Still quite a bit of time/effort, though.

Yeah going halves each time is the fastest way but it took me liek 7 days to bloat it this size on single playthrough... spending another 5x7daays cycles just to track bloating mods (would be progressively harder if there are several of them) would just suck any fun of the game

... I THINK I've heard of a mod causing save bloat issues (Vayra's sector I THINK). ...

And that might be a good one to start with as just a one-off...

Thx! On my next playthrough I'll try without that mod coz save bloating is a pain. Also just by looking at the list of mods I have enabled - do you see any other suspicious ones?

[Alex]

Notepad++ can handle it but I'm not sure I'll be able to find the issue. It's not like each mode is putting it's own stuff in a separate big block
...
Yeah going halves each time is the fastest way but it took me liek 7 days to bloat it this size on single playthrough... spending another 5x7daays cycles just to track bloating mods (would be progressively harder if there are several of them) would just suck any fun of the game

Yeah, not saying it's a good idea by any means, just nothing more effective comes to mind.

Thx! On my next playthrough I'll try without that mod coz save bloating is a pain. Also just by looking at the list of mods I have enabled - do you see any other suspicious ones?

I don't, but this is also the first time I'm hearing about the issue. Others are going to be more aware of the latest issues etc.

(... let me actually move this thread to bug reports (modded); that feels like where it belongs.)

Edit: also modified the thread title in the OP.

[Vayra]

Yeah, this is probably me. Could you upload one of the larger saves for me? I'm not sure where to start diagnosing something like this.

[erikem]

Yeah, this is probably me. Could you upload one of the larger saves for me? I'm not sure where to start diagnosing something like this.

Sure I will. I have just checked and thx to the tidy way you pack stuff in XML I was able to locate ur stuff and check the size. Out of the 270mb of campaign.xml 247mb are indeed taken by your mod.

Here's the campaign.xml I was analyzing: https://www.dropbox.com/s/vsqbpjfaa8hmchx/campaign.xml?dl=0

In order to be able to navigate an XML of this size with ease I was using: http://www.firstobject.com/dn_editor.htm (much faster than Notepad++ on files of this size)

Please have a look at this section of XML (VayraRaiderBaseReaper/hiddenMarkets/ is like the block that does all the bloating coz this block alone is 247mb while the rest of your blocks are less than 1mb):
/CampaignEngine/data.scripts.campaign.bases.VayraRaiderBaseReaper/hiddenMarkets/Market[]/economy/stepper/econ/markets/Market/primaryEntity/orbit/f/where/o/saved - seems all the bloating happens here

[Vayra]

Yeah, this is probably me. Could you upload one of the larger saves for me? I'm not sure where to start diagnosing something like this.

Sure I will. I have just checked and thx to the tidy way you pack stuff in XML I was able to locate ur stuff and check the size. Out of the 270mb of campaign.xml 247mb are indeed taken by your mod.

Here's the campaign.xml I was analyzing: https://www.dropbox.com/s/vsqbpjfaa8hmchx/campaign.xml?dl=0

In order to be able to navigate an XML of this size with ease I was using: http://www.firstobject.com/dn_editor.htm (much faster than Notepad++ on files of this size)

Please have a look at this section of XML (VayraRaiderBaseReaper/hiddenMarkets/ is like the block that does all the bloating coz this block alone is 247mb while the rest of your blocks are less than 1mb):
/CampaignEngine/data.scripts.campaign.bases.VayraRaiderBaseReaper/hiddenMarkets/Market[]/economy/stepper/econ/markets/Market/primaryEntity/orbit/f/where/o/saved - seems all the bloating happens here

Thanks for the detailed info! This helps a ton.