I can answer this one.
I actually colonized two systems with High Danger level remnant infestations, and one system with a low danger level remnant infestation. Terran/Swamp worlds with adundant Rare minerals don't grow on trees after all (just one system had such a gem). Anyway, since I had a colonization task force with no Capital ships, I didn't feel I could stay in system to slug it out with remnant capitals and I wanted to see what would happen with time and neglect. I also added a bunch of industry and commerce before I left both systems to implant Gamma and Beta cores. I also added a System HQ, but did not attempt to build an orbital station (which is where the remnant nexus was located no less).
There seemed to be no deleterious effects economically or militarily from having my colonies co-inhabit a planet with a nexus in orbit around them. I was fighting pirate raids and raids from the Perseans and the Hegemony on my other worlds, so I was occupied for almost half a year fending for those worlds while my co-habitated systems suffered no incursions from any forces other than Ludic Path. I could invest in growth incentives and there was no stability penalty indicated on the colony stability menu either. Both systems reached level 5 in less than six months. I sure wish the Hegemony had looked for some AI cores on those worlds though.
Anyway, I built up a NEXUS removal fleet, and proceeded to visit those colonies being so horribly provided such an effective and completely free defense force, and noticed that Scavenger and Privateer fleets/task forces would spawn and head towards my co-habitated systems when I was within a LY of the colony system. Once I was in system, there were all kinds of overmatched battles where Scavengers, Privateers, and the occasional Patrol force were locked in highly unequal combat. The AI would focus solely on the planetary fast picket forces while I was much closer to them and behind them (also much slower than a fast picket force). It was very easy to then waltz right in and initiate solo combat with the Nexus as all remnant fleets were scattered chasing much faster targets.
It's been a couple days since I destroyed the Nexus and the in system remnant fleets are still powerful, but I jump them when they get isolated, otherwise I steer them away from my colony. No stability impact to the colony, and no observed economical impact from having homicidal remnant AI running loose attacking all shipping. I suspect the remnant fleets will be slowly attrited over time as my forces gain supremacy and the remnants have no means by which to replace material losses.
Based on what I experienced in my play through where I neglected my newly colonized worlds with High Danger remnant forces, I conjecture following:
Primary hypothesis: It is assumed no one would colonize a system with High Danger level remnant forces.
1) Since no one is supposed to colonize infested systems, the AI is not attacking simulated shipping when the player is not within a certain distance of the system.
2) Because of the primary hypothesis, there is no effect on stability.
3) Because of the primary hypothesis, there is no negative economic impact.
4) Because of the primary hypothesis, there is no negative impact on growth.
5) Because the system is still High Danger level, the AI also does not schedule raids by pirates, other factions, or schedule Hegemony inspections.
I also base this hypothesis in part on my experience with the low danger remnant infested system. My player fleet is too slow to catch the enemy droneships even if I intercept the fleet. However, the system fast pickets can catch them. Until I visited this colony, (which is located near by the High Danger remnant colonies), the in system pickets did not intercept these fast (but weak and very finite) remnant forces. After I visited this system, the picket forces were catching and dispatching these small forces.
This lends to the possibility that when the player is absent for long periods (a week or more maybe), the system economic/military simulation becomes abstracted until the player "visits" to make the "in system" simulation start resolving these factors.