What I have issues with however is how hullFrequency works in the faction files: Even with the tiniest values there (such as 0.01) those ships are omnipresent. As in all ship in fleets are "rare" ships with the odd regular vanilla one. It is as if individual hulls that do not have a frequency set in this section are defaulted to 0 rather than 1, and the global blueprint rarity doesn't get applied. And as soon as another mod add their own ships with average frequencies, the relative rarity gets properly reflected in game.
So am I completely mistaken and there is a completely different way to set the spawn rate of a ship in fleets? Or could there be a bug there?
For fleet compositions, the "rarity" column in ship_data has no impact. If a hull is not mentioned in the hullFrequency, it does indeed default to 1.
This is used fairly extensively in vanilla, so, hmm - it's certainly possible there might be a bug, but I remember testing it quite a bit, including with values like 0.0001, and everything seemed to work at the time.
Quick thought - you don't have variant ids specified there, do you? It's looking for hull ids.
One other possibility is to do with ship roles. For example, if you have a single carrier with a low hullFrequency, and no other options for the carrier roles, it'll get picked every time despite its low frequency, since it's the only choice.
Hope this helps!