Something else to consider: with a neural net, you don't really know why it does any one thing it does. So, for example... say there's a relatively innocuous balance change. And in the meantime, the neural net decided that "hull == 3000" is a useful proxy for a ship having some other properties. And changing it to 3001 could, conceivably, break the whole thing horribly.To be fair... Humans express exactly the same level of randomness and lack of logic when looked at as a massed whole. Where a 20/20 review of choices might reveal a far more logical and efficient path you will be hard-pressed to find an actual human in a navigation situation that will pick anything but the path of least percieved resistance even when the path of actual least total effort is easily visible.
More realistically, what happens when there's a new ship? A new ship system? New weapons? I think the ability to handle mods would be... questionable.
(This is all assuming an optimal-or-close-to-it AI would be fun, but that's another question.)
It would hopefully be possible that small stat changes wouldn't matter that much because the actions that you take aren't really different if the ship has slightly more hull or a slightly different load out
Also for most of these AIs (starcraft, dota open AI, etc) the goal is to beat player by playing "smartly". So there are extreme limits on actions per minute. Starcraft pro players are in advantage they get to spam actions. That's the only reason human players can have somewhat competitive game with AI.
An actual example: a neural net trained to tell apart American and Russian tanks actually learned to tell apart low and high quality photos, since the photos of Russian tanks were all low-quality and taken under less-than-ideal conditions. Point being, it may learn to do something, but the *why* is extremely iffy. That's why I'm saying a completely unexpected thing could turn out to in fact be critical. Like, for example, something having exactly 3000 hull, or whatever other random bit of info that's obviously non-critical to a human but a neural net might fixate on for reasons.
I would also love to see AI finding out that unexpected 3 forge strat or something completely out of the box. Something that would force humans to reavaulate the meta.
I'm pretty sure sooner or later deep learning will be part of the game dev's toolkit in some way.
But i don't see many of them saying how much fun they have playing chess or go against AI :))
Hah, there's a lot of hilarious/tragic ways models can go wrong. One of my personal favorites is the general category of Adversarial Objects, specifically: https://www.digitaltrends.com/cool-tech/image-recognition-turtle-rifle/ TL;DR; Google is very sure a turtle is a rifle.
I want to reinforce that a very good AI doesn't necessarily mean the most interesting or fun to play against. For example, if the optimum strategy is to kite the player for 10 minutes with a single ship and then chain deploy until the player's flagship CR runs out then it's going to do that every time. At the end of the day, what we want is an engaging experience and I'd say that the current AI already does that in spades.AI already abuses kiting too much, and I would expect perfect play AI in Starsector to kite even more to play the stall war and win by player running out of PPT and CR first. (It is a major reason why I currently use all-capital or capital and cruiser fleets - maximum PPT to win a stall war.) Something like Remnants, which have more PPT than most ships, could win just by running away and running out the clock. I would like to see Starsector AI rolled back to more aggressive macho pre-0.8a AI. It was more fun.
The ML is still finding real patterns in this case, even if they aren't the intended patterns. There is a knowable 'why' based on the training data, it's just not the 'why' you wanted. The performance can only ever be as good as the data (in my field we say 'garbage in garbage out'). For image recognition specifically, there's going to be a lot of issues with pixel level information/patterns/noise that just aren't an issue in video games where the ML has direct access to the game states and there's pretty much no variance. For instance, if the tank identification ML was acting directly on the dimensions and properties of the tanks rather than on pixel level information representing those things, it's impossible for some of those things to go wrong. It's just sort of hard to compare results from applications with such different data and methods (supervised vs reinforcement learning etc.).It would hopefully be possible that small stat changes wouldn't matter that much because the actions that you take aren't really different if the ship has slightly more hull or a slightly different load out
An actual example: a neural net trained to tell apart American and Russian tanks actually learned to tell apart low and high quality photos, since the photos of Russian tanks were all low-quality and taken under less-than-ideal conditions. Point being, it may learn to do something, but the *why* is extremely iffy. That's why I'm saying a completely unexpected thing could turn out to in fact be critical. Like, for example, something having exactly 3000 hull, or whatever other random bit of info that's obviously non-critical to a human but a neural net might fixate on for reasons.
Based on the games I've seen, though (which is quite a few of them), it wasn't that. I mean, it built roaches into Void Rays and almost lost a game that should've never in a million years been close, that by the end looked like a desynced replay due to the AI's baffling decisions. Its whole zerg "strategy" - at least, what I saw of it - was a really, really, really well executed roach timing.I got the impression from reading the paper on alpha star that the space of possible inputs in starcraft was just so large that reinforcement learning was not able to explore like it does in chess or go. They had to resort to starting it out with supervised learning (basically copying some human gameplay) and then it was allowed to try and improve from there via actual reinforcement learning. I actually noticed that they developed some specialized networks that only did particular strategies, and then tried to make generalized networks that could beat all of those specialized ones, so maybe some of the limited strategies you observed were related to that.
I particularly think the example of hull is not a good one because the ML/NN will naturally see tons of different hull values on tons of different ships over the course of a million training combats. That's all was trying to say: that small variations that are already covered by the training are probably not going to be an issue.
I got the impression from reading the paper on alpha star that the space of possible inputs in starcraft was just so large that reinforcement learning was not able to explore like it does in chess or go. They had to resort to starting it out with supervised learning (basically copying some human gameplay) and then it was allowed to try and improve from there via actual reinforcement learning. I actually noticed that they developed some specialized networks that only did particular strategies, and then tried to make generalized networks that could beat all of those specialized ones, so maybe some of the limited strategies you observed were related to that.