An easy example of this is Safety Overrides. The early- to mid-game is filled with relatively docile and forgiving enemies, so it's easy to just put SO on everything and steamroll through pirate fleets. Thus testing against early- to mid-game fleets will likely result in "use SO". But that is pretty bad advice when it comes to the endgame, when the enemies are much more vicious and when diving headfirst into a fleet means almost certain death. So the player who looks at testing based on the early- to mid-game -- where the results will most likely say "use SO" -- will be in for a rude awakening when he gets to the endgame. Then all the effort put into developing their fleet around SO basically goes to waste.
As an aside, that's why I don't feel like SO needs much of a change -- its power naturally fades as the player encounters harder content. I've noticed that most of the discussion claiming SO is too strong basically centers around a few specific ships -- Hyperion, Monitor, LP Brawler -- or around a player-controlled SO ship, since a human player is much better at maneuvering and managing flux than the AI, so of course a human player can make much better use of SO; or demonstrates it in the sim or against weaker enemies (i.e. "stick SO on ship X and look you can kill ship Y in sim easily"). I've yet to see someone actually show that SO is stronger than non-SO against endgame fleets, say double Ordos, outside of these cases. That's because the opposite is true; a non-SO Eradicator fleet will do much better against double Ordos than an SO Eradicator fleet, for example. So SO naturally becomes a non-factor at the endgame, so I don't see it as needing a change, despite the constant forum complaints about it from certain posters.
Safety Overrides is extremely powerful late game. Eradicator isn't fast enough despite the high base speed and is too focused on armor for safe active venting when SO goes best with a strong 360 front shield. You also save significant OP from not needing PD. Assault Chaingun also isn't very good against high armor and also it's nice to have an all purpose weapon (Heavy Blaster) when you have the flux to support the inefficiency. Small and medium ballistics are usually much nicer than energy slots, but small energy slots are great for close range and medium energy are fantastic for SO.
I don't normally use SO beyond the early game because it's boring and trivializes the content rather than because the power fades. I do agree that an SO flagship fades in relevance. The battles get too long for an SO flagship with a non-SO fleet, and it becomes difficult to safely approach if you're flying the only SO ship. There's also the Odyssey with 2x Plasma Cannon / 3x Sabot SRM Pod and Systems Expertise which plays a very similar role to an SO flagship with full capital tier PPT. Regardless, using SO for the entire fleet gets extremely strong once you get the proper ship for it: the Aurora. It was strong in 0.95.1a and it's substantially stronger in 0.96.
Elite Energy Weapon Mastery
Elite Field Modulation
(Elite) Ordnance Expertise
(Elite) Helmsmanship
Combat Endurance
Target Analysis
2x Heavy Blaster (3 work for anti-armor focus if not using all weapon slots)
1x Pulse Laser
7x IR Pulse Laser (replace front turret with emotional support PD for AI and can drop more for more capacitors)
7x Capacitors
35x Vents
Built-in Hardened Shields
Built-in Hardened Subsystems
Built-in Shield Conversion - Front
Safety Overrides
Solar Shielding
This ship ends up with 0.35 shield efficiency which is even better against Remnants due to Solar Shielding, giving it a ridiculously strong shield from the high flux capacity. Weapon/shield flux is more than covered by flux dissipation thanks to Ordnance Expertise (2736 dissipation) and Elite Energy Weapon Mastery. You can deal with any content in the game with a pure fleet of these with hardly any risk of a loss, or you can use them alongside aggressive Radiant builds. Odyssey works fine as a player flagship alongside them which is nice since flying one of these is very boring.
I usually go with Eradicators, Champions, Conquests, Onslaughts, etc. but now and then it's fun to play with an overpowered aggressive close range fleet. 0.96 has significantly buffed it. The problem is not the Aurora, which is a balanced ship without SO. SO Fury is quite strong too, although with much higher risk of AI losses largely due to the ship system. SO Eagle is also substantially better now thanks to the Eagle buffs (better flux stats and 10 speed are huge). Both SO Eagle and Brawler LP can also now get a full 360 shield via built-in Extended Shields (filling the role of the shield conversion for the Aurora).