How about this:
Size-based limitations:
Size 3: 4 slots available, level 1 buildings only, primary industries only (farming/aquaculture, mining, tech-mining). No market-based expeditions, no pathers, medium pirate raids only with limited consequences (-1 stability, a hit to accessibility for a couple months and losing this month's income, let's say)
Size 4: 8 slots available, level 1 buildings only (except maybe battlestation level 2), all industries available. Small expeditions possible (maybe only mercenaries with no standings hit, like someone suggested), Pather cells but no attacks, medium+ pirate raids
Size 5: 10 slots available, all buildings/upgrades available. No limits on attacks.
Size 6: 12 slots available
Synchrotrons/nanoforges can only be installed in level 2 buildings (new level 2 refinery upgrade needs to be added, buildable at size 5).
AI/tech-mining expeditions as now.
Possibly also restriction of only one building being upgraded or built at a time.
Growth:
Should be tied to planet characteristics more than now (habitability, distance to core, maybe other traits) and possibly require more effort (organizing immigrant convoys/recruitment in the core?, maybe buildings that you can remove once you feel the colony is big enough)
Increasing the size of colonies should be progressively slower and require more investment. 3->4 should happen somewhat slower than now, maybe require some investment if planet is hazardous. Going 4->5 should require significant work/money but eventually be doable for any planet if you wish, 5->6 lots of investment and desirable planet (good accessibility due to proximity to core and/or low hazard rating) and some time, 6->7 heroic efforts, only possible for good planets.
Since growth will mainly be based on immigration, there is a good rationale for fast growth early on and slower growth later, since interstellar transport capacity will be the limiting factor, and requirements will increase by an order of magnitude for each population level.
This would
- Force decisions at the start of a colony (do you want defences, growth, or income?)
- Limit the downsides from overinvesting in a colony early on and limit income from colonies at the start. A built-up colony that gets raided before the player is ready can be a significant money drain right now.
- Encourage starting colonies earlier (since growth takes time and there is less danger) rather than saving up lots of money, colonizing and dumping a ton of cash into building up everything at once
- Make growing a large colony (size 5+) an attractive goal rather than something that eventually just happens to a colony that already does what you want
- Encourage multiple specialized smaller colonies in addition to larger built-up ones
- Keep the existing underlying system largely intact
Downsides (the ones I can think of):
- Increased reliance on finding good planets (in particular habitable planets close to the core) and systems
- Possibly easier to spam smaller colonies
What other problems would this system cause?