I have a few questions:
1. Is there a setting to increase fleet costs for player(and factions as well)? ~600k for 1200DP invasion fleet is practically charity compared to the costs player has to pay for new ships.
Fleet request cost per FP in Luna settings menu, or
fleetRequestCostPerFP in exerelin_config.json
It'd be more expensive if you got to keep the ships afterwards

2. What are the factors that decide AI invasion frequency and strength? I practically have to camp on gates, unable to explore as Heg keeps sending one invasion after another. Granted I am not too kind to their inspection fleets...
3. In general does AI have some kind of fleet budget same way player does? Nex AI functions completely separate to colony crisis system it seems, leading to a very "fun" experience of having to fend off pirate armada on top of regular pirate raid since you are at war with them.
Factions have a pool of invasion points (also used for various other fleet generation events) that generates over time (based significantly on availability of supplies, fuel, marines, heavy armaments and ship parts on markets). The current pool and daily point gain are visible in the intel screen under the faction's diplomacy profile and (if enabled) strategic AI.
The larger and more well-defended a market is, the more they'll spend on any attacks towards it; beating back a major invasion can set them back for a decent time. Though for big factions like the Hegemony you're going to want to be strong yourself, have a powerful ally, or buy them off one way or another.
And yeah, it's not used for any vanilla fleets, crises included. At least a crisis only comes once and can be stalled through several means

(By default pirates don't invade, so won't bother you in that way regardless of hostilities)
4. Are defence fleets useful? Even with boosted high command(alpha core+cryo) I get evenly matched for an average invasion fleet. How can I make sure colonies are well protected for remote battles?
Don't use them much myself, but they've been quite popular, presumably for cases where the player doesn't want to go back and deal with it.
do npcs adhere to their own tolerance for saturation bombardments? I noticed theres a setting that lets npcs use them and wonder if they will use it if they themselves turn hostile if another faction uses it.
There's currently no relations penalties for an NPC faction doing sat bombs. I could add one, but then they'd need to be taught discretion about using it in the first place (also there's currently no way to detect sat bombing from vanilla features like Pather or Diktat crisis).
How does strategic AI differ from regular AI? Does it affect battles also? If I find it too brutal, can I just turn it off from the mod settings window?
It makes factions do things like invasions, industry construction and diplomacy based on a goal-directed system.
This old dev screenshot gives the idea.
Doesn't affect tactical battles.
Can be turned on or off midgame at will, although you'll need to save and reload for it to apply.