First of all, thank you guys so much! I've given your mentions trial, but didn't work out well.
I tried to set the game to use my dedicated GPU, but it seems that the game is using BOTH of my GPUs according to monitoring. Not a clue what is happening here.
CPU monitoring doesn't provide much information, all cores seem to peak their usage randomly, while none is obviously the limiting factor.
I've also tried to update my Java, not really any changes.
2 new leads I have notice, 1st is that the performance issue seems to be the worst in hyperspace. 2nd is that for some reason, my game lag the least when assigned VRM is about 3GB, increasing or decreasing it both cause more lag. Despite that I have 4+0.5GB GPU ram and 15.4GB total ram.