Sorry about the trouble! Can you try running using starsector.bat instead of the .exe? It should be located in:
C:\Program Files (x86)\Fractal Softworks\Starsector\starsector-core
If that also fails, could you try running the .bat file from the command line, to see what error it gives you? That should be more informative than the error from the .exe.
To do this:
1. Windows key + R, type in "cmd" (without the quotes) and press enter
2. cd "C:\Program Files (x86)\Fractal Softworks\Starsector\starsector-core" (this time with the quotes; press enter)
3. starsector.bat and press enter
The likely culprit is the _JAVA_OPTIONS variable being set on your system, in which case removing it from your environment variables would resolve the issue - in fact, I'd just try looking for that first, before even doing these steps - but it might also be some other sort of problem.