There is a nice
Mods Index that can be found on here that has a list of some mods. It has some utilities mods with particularly useful being the common radar that adds a mini radar map both in map and while in battle. There are other utilities too.
The two basic Major Mods are Starsector+ and Nexerelin.
Starsector+ enhances the game and adds a lot of cool pirate ships. It also balances the game in a way that when used in conjuction with the mods it supports (found on it's page) no faction will be too overpowered compared to the other and the game will not feel like one faction is weak and the other all powerful. Well Knights Templar are overpowered, but they are meant to be that way.
Nexerelin changes the way the game play, making it so factions can take over markets and the battle feels more real, as the galaxy changes instead of the factiosn always having the same planets. You can either play in Corvus mode, where you play in the original Starsector galaxy, but with the Nexerelin rules, or you can create a random universe with customized number of stars/planets/stations and play in there, which gives a great replay value. On Nexerelin Faq you can fine which mod factions are supported and which of them support Corvus mode.
My first game was getting factions that were both compatible with SS+ and Nexerelin Corvus mode and played in the original galaxy with some extra factions that were compatible with both mods.
If you do not care to play in the original galaxy, just get what mod factions you want and add them, creating a nice universe with many different faction.
Currently I am playing on an older Starsector version (0.65.2a) with Nexerelin, Starsector and 24 mod factions. Chaotic and nice
Just to point out an observation, having many factions does not seem to make the game slow down much. Both my gameplays with some some factions and now 24, have been in a random galaxy with max stars/planets/stations. The one does not seem to be slower than the other.
Since Jude suggest
Ironclads, I will tell you to not get it for a while but keep an eye until the version changes from 10.1. I played it a bit and it was awesome. Awesomely hard, totally different from the normal game or any other mod and generally awesome. Sadly currently the bounties on it are broken, so it is quite hard to make money only from bounties and it is also hard to raise reputation with factions, since you can;t go around killing their enemies and make them like you. The creator seems to be lost about it, but hopefully he will manage to make it work.