If cores are not an option, then I put priority on self-sufficiency and Pather cell avoidance. On my four worlds in my last game, I had...
* Gas Giant with Mining (for volatiles), Light Industry, and Heavy Industry (no forge due to pather cells)
* Tundra world with Farming, Fuel Production, and Mining.
* Random habitable with Farming, Heavy Industry (with forge), and some other no interest industry.
* High-gravity habitable with Farming, Mining, and Refinery.
All four had the obligatory High Command for colony defense.
When avoiding Pather cells, items in various industries put interest at the safe max, which really limits industries that can be placed without attracting the zombie Pathers.
I had extra colony slots for pop-up colonies here and there for temporary base of operations, then eventually built a fifth colony near a red system for Ordos farming.
I got max colony skills in that game because wanted an empire, and I did not know about the Pather bug until much later, and by then, I did not feel like restarting. I missed combat skills due to sunk points in colonies, but I wanted my empire! With alpha cores as they are, I would totally dump Industry for more combat skills. (I gave up officer management and armor skills for colony skills.)
As for pollution, a low hazard world with pollution probably has less hazard than a non-habitable world (which probably has 125% or 150% hazard minimum). Pollution is less severe than Decivilized, which puts a big stability penalty on top of +25% hazard both give.