After my first play through where I rushed special modification. I really don't think it's worth it until fairly late in the game. I think I had 3 ships with a third S mod for most of the game. Although the patch increasing story point generation happened in the middle so it's hard to say exactly how that worked out. I think I will take the AI skill next time early, and spec out of it late.
I would not rush it either because I probably do not have all of the good hullmods I want builtin when I can learn Spec.Mods. I lucked out with Missile Racks early, but I did not get the rest until late in the game.
But late in the game when I have everything I need and points to burn, I would go nuts with s-mods if I have Spec.Mods.
Love most things in this patch, but the DP and officer stuff is really dumb. I cannot wait for a mod to change it because it's really making late game miserable and if you were still using a frigate or dessie anywhere. Also SP is flawed, you cannot accrue more fast enough to fight, and while mercs are a cool idea the cost of them in sp form is not sustainable. Fighting one fight then saying "well guess the next time i'll just be ***" is dumb.
Colony improvement costs mean that any spare points get hoarded then eaten by colonies if I do not need them for s-mods.
P.S. Speaking of colonies, improvements plus Industrial Planning means it is possible to meet demand without items. Nice for cramming (Heavy Industry and Fuel Production) industries on one planet without aggravating the Pathers (since classic items add +4 to interest). Lower ship quality is tolerable if infrequent enough for Field Repairs to remove d-mods.