Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Johnny Cocas]

Hi! I'm new to the forums as I saw the need to say something about a problem that has been haunting me for a long time...

Occasionally my game crashes and throws a "GC Overhead limit exceeded" error like many people before me.
The problem is I have done the 64bit java replacement, increased the max memory that the game can allocate and the game still uses A LOT of memory, more than what it was supposed to. Like they said, 4GB is more than the game should ever use.
I have tried everything available on the forums, tried a lot of stuff related to modifying the way GC works, but with no effect... The game still crashed and used strange amounts of ram in every attempt to fix the issue.

The game uses around 2.2-2.6 Gb of ram when I start the game, load the save and start roaming, but as soon as I hit the "Save" button (both Save and Save Copy have the same issue) the ram the game is using climbs to 4.3-4.8 Gb. AND STAYS THERE. Over time it decreases to ~3.8 Gb but once I save again the game consumes another 1gb of ram for no reason...

This is my vmparams file:

Code

-server -XX:CompilerThreadPriority=1 -XX:+CompilerThreadHintNoPreempt -Djava.library.path=native\\windows -Xms2048m -Xmx4096m -Xss1024k -classpath janino.jar;commons-compiler.jar;commons-compiler-jdk.jar;starfarer.api.jar;starfarer_obf.jar;jogg-0.0.7.jar;jorbis-0.0.15.jar;json.jar;lwjgl.jar;jinput.jar;log4j-1.2.9.jar;lwjgl_util.jar;fs.sound_obf.jar;fs.common_obf.jar;xstream-1.4.2.jar -Dcom.fs.starfarer.settings.paths.saves=..\\saves -Dcom.fs.starfarer.settings.paths.screenshots=..\\screenshots -Dcom.fs.starfarer.settings.paths.mods=..\\mods -Dcom.fs.starfarer.settings.paths.logs=. com.fs.starfarer.StarfarerLauncher

And my mod list:

Code

Audio Plus
Blackrock Drive Yards
Combat Alarm Sounds
Combat Chatter
Common Radar
Console Commands
Diable Avionics
DynaSector
GraphicsLib
Interstellar Imperium
Junk Pirates
LazyLib
Leading Pip
Neutrino Corp
Nexerelin
PBC
Extra Salvage/Boarding
P9
Portrait Pack
Save Transfer
SCY
ShadowYards
Ship and Weapon Pack
Starsector Plus
Steiner Foundation
Tiandong Heavy Industries
TorchShips (adds evasive maneuver "burn" to campaign)
Tore Up Plenty
TwigLib
Underworld
Upgraded Rotary Weapons
Version Checker

The list is pretty big, and some mods are probably incompatible, but the only reason given for my game to crash is GC overhead exception.
I'd be really lucky is the problem was just an incompatible or unstable mod in my mod list, but I've checked the whole forum for anything related to GC and failed to find anything useful as most times the problem is poor memory limits assignment or executing the wrong file (.exe vs .bat and such).

I am using a laptop, with the following hardware:
Processor: i7-4510U
GPU: NVidia Geforce 840m (2gb)
Ram: 8GB

Running the game at 1366x768 with the max amount of systems, planets and stations when starting a game (pretty heavy start, but still...).
I don't know what else to say or what more information I can provide...
I tried to analyse one of the crash dumps a while back (3.7GB file, wow xD) and the memory hog is related to the campaign, can't really tell exactly to what as I eventually reinstalled the game and have no dumps/screenshots to provide.

If you need anything else please do ask, I know 0.8 is almost around the corner but this really bothers me because when I save I am constantly thinking "Will my game crash now or will I be able to save?".
It can be a minor issue like me being really dumb or blind and not spot something really obvious, but I seriously cannot see it

Thanks in advance for any help anyone can provide me.


EDIT:
Now I am really ashamed...
I just noticed that the game is loading with, and I quote:

Code

2    [main] INFO  com.fs.starfarer.StarfarerLauncher  - Java version: 1.8.0_91 (64-bit)

It has been using Java 8 all this time for some reason...
So now my question is, does this have anything to do with the occasional crash?


EDIT 2:
Tested the game again with Java 7 (64bit) and when saving the ram used went from 1.8gb to 4.7gb, decreasing to 3.4gb as soon as the game finished saving and further down to 3.1gb after cruising around for a bit.

[MesoTroniK]

JRE8 is not the issue, though it is still highly advised to use JRE7 but...

Running the game at 1366x768 with the max amount of systems, planets and stations when starting a game (pretty heavy start, but still...).

With your mod list, you need like 6 gigs of ram allocated for a Corvus Mode Nex game... And with a Random mode Nex game with maxed settings like you are doing you likely need closer to 8 gigs.

[Johnny Cocas]

With your mod list, you need like 6 gigs of ram allocated for a Corvus Mode Nex game... And with a Random mode Nex game with maxed settings like you are doing you likely need closer to 8 gigs.

So the issue can still be the need for more memory ehh? I can try higher memory settings and/or play with lower amount of systems, but is it normal for the game to go from 1.8gb to 4.7gb of used ram just by pressing the save button? If I keep the game running without saving it never goes over ~2gb... More memory surely will not fix this problem, right?

[Midnight Kitsune]

Another note is to drop the Extra Salvage/ Boarding as it is a buggy mod and might be adding much more strain than what it should

[Johnny Cocas]

Another note is to drop the Extra Salvage/ Boarding as it is a buggy mod and might be adding much more strain than what it should

Humm, ok. This mod will not be missed that much
Any idea to what is causing that massive burst of memory usage?

[Midnight Kitsune]

Another note is to drop the Extra Salvage/ Boarding as it is a buggy mod and might be adding much more strain than what it should

Humm, ok. This mod will not be missed that much
Any idea to what is causing that massive burst of memory usage?

Maybe the setting of 2 and 4 gigs instead of 4 and 4 gigs. That should keep the mem from spiking and it is suggested that you keep them the same so that Java doesn't need to scramble to free up more space

[Tartiflette]

Given the amount of systems that will be present in vanilla in 0.8, even if most will be quite barren, I suspect the memory issue had to be somewhat addressed.

[Johnny Cocas]

I wil try to increase the minimum memory to equal the max setting and see what happens, but it still puzzles me why on earth does the game need 3gb of extra ram JUST TO SAVE like I experienced last time I played the game.
Like I said, even though I am playing at the max nexerelin settings and am using GraphicsLib with almost everything activated the game is always around 2gb of used ram IF I never hit the save button.

Something must be causing a memory leak in the act of saving, but how can I find what is causing it? I am willing to try everything even if I have to analyse or type code on my own, can't be worse than my Java classes

[TJJ]

Starsector uses a DOM parser for reading & writing its save files.

DOM parsers hold the entire object model in memory at once.
So the game is copying the entire campaign object graph into a 2nd general purpose (i.e. less efficient) object graph, and only then serializing it to disk. (naturally it does the same, but in reverse, when loading save games).
This duplication of the object graph data results in a massive memory spike.

It's not such a problem in vanilla 0.7.2, but as most mods tend to (massively) expand the size of the campaign object graph, the limitations of the current code become much more apparent.

Switching over to a SAX parser (event driven) would be the way to go; they're less flexible (in particular highly-interconnected data can be troublesome), but would avoid the duplication of the object graph and thus drastically reduces peak memory usage.

[Dark.Revenant]

Another way about it is to reduce the problem space and save logical chunks of the campaign into separate objects.  This would, at worst, reduce the memory spike, and potentially allow faster saves if the chunks could be arranged in such a way that parts of the campaign could be outright ignored based on context.  It likely requires less work than switching design paradigms, too.

[Johnny Cocas]

Thanks for explaining the mechanics behind it.

Those are unfortunate news though... Has this possible change been discussed with Alex? I know Dark.Revenant even used a java debugging tool and found out that there was an issue with campaign being duplicate or something but I didn't understand much and didnt get if the issue has been solved or not, the discussion kinda of stopped there.