Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Vanshilar]

Yeah for Harpoons, I'll probably end up testing it both ways.

The thing is, the AI tends to fire Harpoons when the enemy target is at high flux. But the Conquest has so much other DPS that by the time the Harpoons arrive, the enemy ship is already dead or nearly so, so it results in a lot of wasted missiles. So it may be better for the Harpoons to be fired with the Squalls, so that by the time they arrive, the enemy ship is just reaching high flux, perfect for a missile volley. So I'll try it both ways, and base that decision on which way results in a higher hit rate and/or more overall damage.

Yeah the "traitor" command works well to stop the end of combat. So going forward, that's how the test will be, i.e. me in Medusa grabbing one objective and then helping the battle run smoothly, and a Gryphon at the other objective. Then the middle starts with 4 Conquests, with a 5th one joining in after I get both objectives. The Conquests also end up doing around 80% of the damage (instead of before when they were doing around 60% of the damage since I was hero-ing it up in my Onslaught running into big ships with Prox and thus grabbing the damage), so it'll be more representative of how the Conquests do on their own. On the final Radiant, when it's about to die, I'll use the "traitor" command on it to get it to switch sides, turn on god mode so the remaining shots don't kill it (although that slightly decreases everyone's hit rate), then count up the ammo on each of the ships to see how many were fired.

[Thaago]

Re: the hit rate of weapons tested with ammo and detailed combat results:

That is really interesting data! The HAC vs HVD especially because it answers a question I've long had: does the HVD accuracy make up or more than make up for its lower max DPS? Answer from your data is no, the HVD is still lower DPS in practice. But it has many other advantages so I still see it as a good gun, just not the always better gun (hooray!).

For Gauss's, in my experience their hit can suffer from the turning rate of the mounting ship being too high, so I nearly always install advanced turret gyros (I'm weird in that I consider it to be a good hullmod - not right for every ship but important when needed). They also are sometimes just a victim of their extreme range and shoot at fast targets that are very far away. Faster projectile velocity helps but not always.

[Hiruma Kai]

Re: the hit rate of weapons tested with ammo and detailed combat results:

That is really interesting data! The HAC vs HVD especially because it answers a question I've long had: does the HVD accuracy make up or more than make up for its lower max DPS? Answer from your data is no, the HVD is still lower DPS in practice. But it has many other advantages so I still see it as a good gun, just not the always better gun (hooray!).

I think you might be forgetting to account for uptime and the fact shots become harder the farther out the target is.

Naive calculation just looking at hit rate:
0.8091 * 138 DPS for HVD = 111.6 DPS for HVD
0.5926 * 214 DPS for HAC = 126.8 DPS for HAC

I'm guessing (correct me if I'm wrong) that is the calculation you did make the statement the HVD accuracy doesn't make up for the DPS difference.

However, the HVD was firing at targets the HAC didn't even bother to try shooting.

HVD had 639 shots/30 shots per minute = 21.3 minutes worth of fire
HAC had 783 shots/(129/3 shots per minute) = 18.2 minutes worth of fire.

Presumably, that means roughly 3 minutes of fire was in the range band of 800 to 1000 (times skill and ITU range bonuses I'm guessing), which would be the range at which the HVD has it's worst accuracy, given the target has the most time to change velocity.  So presumably it's accuracy in the 800 and under range band that matches the HAC is even higher than presented here.

Edit: Although re-reading, those wouldn't have been on the same ship at the same time, so making the assumption the range profiles were identical between runs is poor, unless it was a mix of HVD and HAC simultaneously?

Ideally, what you'd want is the accuracy of the HVD vs HAC within the 800 range band, not the 1000 range band vs the 800 range band.  It's also likely, that at very close ranges, it actually favors the HAC significantly, as the accuracy at a range like 100 is likely to be close to 100% for both weapons.

What is better, and typically reported by the Detailed Combat Reports mod, is in fact the total damage dealt by the weapon system in actual field combat situations, so that the ranges tested, are the ranges actually experienced.

If you make the assumption that Vanshilar's piloting produces a typical set of ranges of engagement for most ships for each attempt, we can consider the total damage dealt.
HVD: 517*275 = 142,175 kinetic damage (with 137.5 armor pen) + some ion damage
HAC: 464*300 = 139,200  (with 50 armor pen)

That looks like a win for the HVD to me, based on this data sample.  Which is kind of what you expect for a higher OP cost and less efficient weapon.

If Vanshilar's piloting does not match range for engagements between runs, then it's hard to reach a general conclusion.

Edit:
Probably the best way to answer the question on whether the accuracy makes up for the DPS is to mod the max range of the HVD to be the same as the HAC, modify a ship to have two medium ballistic mounting points directly on top of each other (so same position on the ship with same arc), put a HVD in one and a HAC in the other, and run that through a couple battles keeping track of total damage for each.

Edit 2:
Although Vanshilar's second data set is close to what you'd want now that I think about it, but it's still splitting it up into multiple runs, but hopefully the averaging of many ships approaches some kind of gaussian distribution of engagement ranges.  I'll note the Arabelest, range 700 base, had only a 20% uptime compared to the HVDs 53% 9both relative to the Squall's 100%.  Although different weapon configurations with different ranges are going to yield different average distances the ships tend to stay at, so might have non-trivial variance between runs.  As I note, ideally putting all the weapons on a single mount in the same spot with the same range is going to be the fairest in terms of trying to determine the accuracy within a particular range band.

[CapnHector]

This is a highly interesting and relevant question. Technically it might not be exactly equivalent to mod the maximum range of the HVD to the same range as the HAC as, as you noted, closer ranges favor the HAC. This is because we might conceptualize of where each shot hits for each weapon as a sum of a uniform distribution (shot angle) and a normal distribution (error in position compared to the target being in the center of the field of fire and not moving). Now a perfectly accurate weapon only experiences the error in position component, so its chance to hit is only determined by the error in position component. But the error in position should reasonably be expected to be a function of range since, due to projectile travel time, the target vessel's position has more time to change when it is further away.

So there are two different sorts of error and the question is how they will compound. How will this work out in practice? Well, let's use the idea from the simulation that SD(positional error) ~ range and a factor f such that SD = f*range. Look only at the 6th shot onward so that the maximum angle of the HAC is 18 degrees. The HVD is always perfectly accurate. Then using the script I posted above we get this

Range Hit chance (vs Dominator, width 220 px, to hit ship)
     HAC    HVD
100  1      1
200  1      1
300  99.7%  1
400  97.4%  99.99%
500  94.1%  99.9%
600  90.7%  99.4%
700  87.2%  98.5%
800  83.8%  94.7%
900  80.9%  96.2%
1000 78.1%  94.9%
1100 75.8%  93.4%
1200 73.9%  91.8%
1300 72.0%  90.3%
1400 70.5%  88.8%
1500 68.9%  87.6%

So it is not quite the same to engage at a HAC at long range as it is to engage with a HVD at short range, as if we set HVD's range to 700 we have a 11.3% point difference in accuracy whereas if we set HAC's range to 1000 it is a difference of 16.8% points. Of course, we do not know what the actual law to describe the standard error adequately is. Maybe it's SD ~ range^2. In such a case the results would be different. Although since travel time is linear and the enemy ship's maximum travel is presumably a linear function of time, I would go with the linear assumption.

Now I would assume that the actual average engagement range is determined by how the AI behaves given a particular set of weapons, making the issue highly complex. Vanshilar's simulation results do suggest that in real combat the ship is not engaging optimally with the Arbalest, likely making it an inferior choice at least in this weapon layout.

[Vanshilar]

I'm still collecting data from testing and not yet ready to release the results (wanted to run them several times to make sure they weren't anomalous), but a couple of notes so far:

1. Locust firing more often than Squall: Looks like this is just because when there are enemy fighters around, only the Locusts will fire. Both are in the same group and linked, but if the AI Conquest has another weapon group active, then the Squall/Locust group is on autofire, in which case only Locusts will fire if there are enemy fighters but not enemy ships nearby. (I did add the enemy fighters back in for my new testing, since that'll give a more accurate representation of actual fights. However, it means that weapon hit rates are lower because some of those shots go toward enemy fighters instead of toward enemy ships.) So I'll still use the Squall as the yardstick for how often the other weapons fire, even though the Locust's rate exceeds 100%.

2. Harpoon being in same group as Squall (or Locust) doesn't really seem to change its hit rate. However, when it's in its own weapon group, it fires much less frequently...as in, sometimes the battle will end and the Conquests still have Harpoons left unused (this is against double Ordos). So it depends more on if you want all the Harpoons used in the first minute of combat, or gradually throughout the battle. I think that because the hit rate doesn't really change, it's probably better to have it be in the same group. The first reason is to ensure that you get the maximum benefit out of them. But the other reason is that during that first minute, you're still trying to deploy your forces, and your reinforcements from capturing objectives are still making their way to the frontlines from the bottom of the map, so that's when you're the weakest relative to the enemy fleet. So that's not a bad time to have maximum damage output.

The HAC vs HVD especially because it answers a question I've long had: does the HVD accuracy make up or more than make up for its lower max DPS?

Yeah, HAC vs HVD was one big reason why I wanted to look into this more. From using Onslaughts with both weapons, they both do about the same overall damage per weapon throughout the fight. Since the HVD has longer range but is less flux efficient, I wasn't sure if it was because the longer range meant it was firing more often, or if it meant that the HAC missed more often, for the two to end up with similar overall damage. I wanted to look at Gauss vs Mjolnir for a similar reason.

Probably the best way to answer the question on whether the accuracy makes up for the DPS is to mod the max range of the HVD to be the same as the HAC, modify a ship to have two medium ballistic mounting points directly on top of each other (so same position on the ship with same arc), put a HVD in one and a HAC in the other, and run that through a couple battles keeping track of total damage for each.

I don't have time to respond to all of it right now but I did test pretty much this way for the Gauss, on my manually piloted testing Legion: basically multiple mounts right next to each other in the middle (though not directly on top of each other because then I can't select them in the refit screen, but separated by 4 units) in a fleet of Gryphons. When Gauss's range was set to 900, Mjolnir's hit rate was around 80% but Gauss's hit rate was around 64%, so yes I think it comes down to Gauss's very low turn rate. It probably didn't help that since I'm manually piloting the ship and will start turning ahead toward the next target before I'm done with the current one, I'm basically turning a lot. This was also how I tested the weapons for their hit rate a couple months ago here, by putting the weapons almost right on top of each other and then comparing how much damage they did, although I didn't equalize the weapon ranges in that case.

AI-controlled Conquests however don't do that as much, so the Gauss's hit rate is only roughly 10% worse than that of the Mjolnir even though the Gauss has longer range. I don't think the extra distance is the reason for missing more though; the Gauss's base projectile speed is 1200, becomes 1600 due to Ballistic Mastery, and the difference between 900 base range and 1200 base range is 2220 - 1665 = 555 SU. The Gauss's projectile covers that in 1/3 of a second, and the target likely won't have much moved in so short a time for it to miss. So I think it's almost certainly due to the turn rate.

My next batch of data will be based on how the AI-controlled Conquests perform, so it won't be dependent on how I personally pilot the ship.

[CapnHector]

Turn rate! Oh, why didn't I think of that.

So, another interesting non-linearity is that if the target vessel moves 100 px at a range of 100 then this equals a change in weapon angle of arctan(100/100)=45 degrees. However, if it moves 100 px at a range of 1000 then this equals a change in weapon angle of about 5.71 degrees to still be aimed at the target.

So if the weapon has a turn rate of 3 (Gauss Cannon) or 10 (HVD) rather than 25 (Mjolnir) this can be expected to make quite a bit of difference if that rate is in degrees / sec and the enemy is close. The turn rate of the HAC is 7, though, so this may not be particularly relevant to the HVD / HAC comparison.

Probably not much of a way to coherently model this other than as another weapon specific normally distributed error.

Edit to add: as a result of considering this, I simulated a few combats of a Gauss Conquest vs. SIM Enforcer x 2 Sunder x 2 Hammerhead x 2 under AI control (the Conquest wins, of course). Definitely could observe issues in Gauss Cannon turning, although there were also obvious misses even when the Gauss Cannon was rotated correctly. I am for now certainly adding Advanced Turret Gyros to my Gauss Conquests until it can be ruled out that it is an issue. Here is my current optimized "generalist Conquest" in burn 10 configuration suitable for space piracy.

[Schwartz]

This is an exciting run-down. Please, if you can, include 4x Railgun with Ballistic Rangefinder for the Medium slots (same cost as 4x HVD, same range, better damage / flux) since Mjolnir / Locusts / Railguns is my Conquest layout and it's a stellar performer.

[Thaago]

...Presumably, that means roughly 3 minutes of fire was in the range band of 800 to 1000 (times skill and ITU range bonuses I'm guessing), which would be the range at which the HVD has it's worst accuracy, given the target has the most time to change velocity.  So presumably it's accuracy in the 800 and under range band that matches the HAC is even higher than presented here.
...

Really good point (and the rest as well, though this stood out to me)! I hadn't considered that the accuracy is absolutely going to be non-uniform across the range for HVD vs HAC, even though I'd just mentioned how it goes down for Gauss.

...Yeah, HAC vs HVD was one big reason why I wanted to look into this more. From using Onslaughts with both weapons, they both do about the same overall damage per weapon throughout the fight. Since the HVD has longer range but is less flux efficient, I wasn't sure if it was because the longer range meant it was firing more often, or if it meant that the HAC missed more often, for the two to end up with similar overall damage. I wanted to look at Gauss vs Mjolnir for a similar reason...

And for an Onslaught in particular the higher flux costs of the HVD can be a real sticking point - less so for the Conquest with its massive flux of course. I would love to see some of your data duplicated with the HVD's lowered to 800 range; they would be less effective overall weapons, but getting the "true" accuracy at 800 range would be good data to have. This would also be a good data point for CapnHector: as they pointed out they have an accuracy distribution in the model based on reasonable assumptions, but we don't have too many data points to really dial that accuracy model in.

Thanks to both of you yet again for doing so much work here! its really fascinating to see the results and I've taken a whole bunch of notes of builds to try based on it.

[Liral]

Whereas this study confirms the many years Alex has spent balancing vanilla Starsector ship and weapon choices to reward skill and strategy—as I suspect many content modders only wish they could for want of time and feedback—packaging the code behind this mod into a tool that would sort combinations by time-to-kill and report results to a spreadsheet or Console Command would enable these modders to highlight imbalances whether potential or long-hidden.  I would gladly contribute to the effort.

[CapnHector]

Whereas this study confirms the many years Alex has spent balancing vanilla Starsector ship and weapon choices to reward skill and strategy%u2014as I suspect many content modders only wish they could for want of time and feedback%u2014packaging the code behind this mod into a tool that would sort combinations by time-to-kill and report results to a spreadsheet or Console Command would enable these modders to highlight imbalances whether potential or long-hidden.  I would gladly contribute to the effort.

Unfortunately I have absolutely no clue about programming other than math and statistics. Although I should probably learn at some point as I would like to make a mod where every faction has their own version of the Conquest.

However, R is free and commonly used software, and this script is configured to accept arbitrary weapons. So if you wanted to test, say, a gun that fires 1 shot dealing 1000 HE damage every 5 seconds, and has a max spread of 1 on the first shot, 2 on the second, etc. to 5, then you would input this into the script

Spoiler

#vectors in R are written as c(value1, value2...)
#the super gun fires 1 shot on second 1, then takes 4 additional seconds to fire again
superguntics <- c(1,0,0,0,0)
#accuracy for the first shot is 1, increasing by 1 to a maximum of 5
supergunacc <- c(1,2,3,4,5)
#damage per shot, damage type (2=kinetic, 0.5=he, 0.25=frag, 1=energy), tics, weapon name, weapon accuracy over time, hit chance
#defines a data structure containing all the attributes of a weapon
supergun <- list(1000, 0.5, superguntics, "Super gun", supergunacc)

[close]

And then add this to the weapon selections, assuming the super gun is a large gun:

weapon5choices <- list(gauss, markix, mjolnir, hellbore, hephaestus, stormneedler, supergun)
weapon6choices <- list(gauss, markix, mjolnir, hellbore, hephaestus, stormneedler, supergun)

(the weapon choices are 1-2: large missiles, 3-4: medium missiles, 5-6: large guns, 7-8: medium guns by default - but it does not really matter at all, from the perspective of the code the choice of which weapons to compare in which slot is completely arbitrary, input "dummy" to have an option of having no weapon in that slot)

Finally, we must re-generate the lookup table for the accuracy distribution

generatelookuptable <- 1

So that should do it. First run weaponsoptimize3 and then weaponsaggregatemeantimebasic2. The script will run and write the tables in txt form into the user's working directory. There is a problem where there will not be a complete table for slots 3-4 (medium missiles) which I haven't had time to fix, only data for individual weapons.

Note that the script takes a very long time to run, due to the sheer number of combinations to be tested and tables to be generated, about a day on my computer. This might be clumsy, so I'll include as an attachment to this post the script providing time series visualization. This script will instead produce a chart upon manual input of weapon choice, like so:


Now if anybody wants to actually try running these scripts, just ask if you encounter any problems. Also, they are mostly just crunching straightforward math, so I imagine they will be easily legible to an actual programmer.

Remember when using that range and turn rate are ignored (other than as the f-factor) but are important in actual balancing.

[attachment deleted by admin]

[Liral]

Unfortunately I have absolutely no clue about programming other than math and statistics. Although I should probably learn at some point as I would like to make a mod where every faction has their own version of the Conquest.

Fortunately, I know math, stats, and programming!

However, R is free and commonly used software, and this script is configured to accept arbitrary weapons. So if you wanted to test, say, a gun that fires 1 shot dealing 1000 HE damage every 5 seconds, and has a max spread of 1 on the first shot, 2 on the second, etc. to 5, then you would input this into the script

Sounds like an occasion for a GitHub!

Note that the script takes a very long time to run, due to the sheer number of combinations to be tested and tables to be generated, about a day on my computer. This might be clumsy, so I'll include as an attachment to this post the script providing time series visualization. This script will instead produce a chart upon manual input of weapon choice, like so:

I wonder if we could reduce the time to get the answer we want from the question we have ("Is it balanced?") from running the script.  In order: simplify the problem, profile the script to find any performance sink, optimize any intensive part found, and finally scale-up the computational resources.

Now if anybody wants to actually try running these scripts, just ask if you encounter any problems. Also, they are mostly just crunching straightforward math, so I imagine they will be easily legible to an actual programmer.

Remember when using that range and turn rate are ignored (other than as the f-factor) but are important in actual balancing.

Let me see the code! 

[CapnHector]

Very likely there are huge performance sinks as I have not attempted to optimize at all, since I presumed to only run this a few times. Code is in attachments to message above and in message #2 of thread.

One more issue is there is currently no hard-soft flux consideration. This is just a matter of adding an extra parameter to keep track of how much "shield hp" can be "regenerated" without lowering shields. So, that would be needed if beam weapons were included.

[LinWasTaken]

mfw i just max out shield efficiency and dissipation with some flux efficient weapons.
meanwhile other people here are calculating damage against armor, projectile speed, etc...

[CapnHector]

Conquest: the (over)thinking man's ship

[Liral]

Very likely there are huge performance sinks as I have not attempted to optimize at all, since I presumed to only run this a few times. Code is in attachments to message above and in message #2 of thread.

Ah, thanks!  I now see that before I could reduce the execution time of the code, I would need your help to understand its structure and function and accordingly reorganize the project, which is in three big files, each of which calls its functions in one global scope; each of those functions is individually commented, but their relationships within the file and the file itself are not.  I would have to first divide the files into smaller files, each one containing the functions for each purpose,  then consolidate into sub-routines the 'driving' code that calls these functions, and finally write one main routine calling these sub-routines.  I could then find steps that could be saved, profile the performance of the code to find time sinks, optimize the intensive portions, etc.

One more issue is there is currently no hard-soft flux consideration. This is just a matter of adding an extra parameter to keep track of how much "shield hp" can be "regenerated" without lowering shields. So, that would be needed if beam weapons were included.

Nothing like adding features!