Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Goumindong]

. That doesn't mean a world model doesn't exist - some basic experimentation will show that it *does* exist, it's just alien and illegible.

No actually. Basic experimentation shows that the world model does not exist*. This is not a surprise if you understand how correlation works.

*the easiest way to do this is actually image recognition. These models fail in ways which cannot be solved by more data because of how correlation engines exist. They can achieve high success rates but the failures are systemic and obvious. They also fail in counting (because correlations on bit data turn hard numbers over the whole into a probabilistic distribution).

 The system only knows what the model images look like when they label something (probabilistic count dependent). But doesn’t even really “know” that.

[iguanaman]

It understands the concept of a tree and guides the noise to something close to what it knows as a tree

So… this isn’t true. Wikipedia says that it lists “concepts”. But it does not.

I don't think getting into semantics helps here.  Sure it's important to be precise, but "concept" is used fairly regularly when talking about this technology

Results,
shown in Figure 3, confirm that the tested neuron indeed learns the concept of faces.

Building high-level features using large scale unsupervised learning
https://arxiv.org/abs/1112.6209 (emphasis mine)

When interpreted this way, the image encoder is the computer vision backbone which computes a feature representation for the image and the text encoder is a hypernetwork (Ha et al., 2016) which generates the weights of a linear classifier based on the text specifying the visual concepts that the classes represent .

Learning Transferable Visual Models From Natural Language Supervision
https://arxiv.org/abs/2103.00020 (emphasis mine)

It's not learning a concept in that it understands empathy, or hatred as a person would.  It's learning the visual concept of a cat, or a rocket.  As we know images are not stored in the model, instead "concepts" (for lack of a better word) such as tree, which you can enter into the prompt to get the models "interpretation" of that by "guiding" the noise.  Concept fits well here because it doesn't always correspond to an object (tree), but also something like "dreamlike" or "blurred".

I think we both agree that the "AI" (which as previously mentioned in this thread is a bad term for it, given it's lack intelligence) doesn't work similarly to a human brain, and simply produces output that we would expect from it's methods.  Yet I think it's fair to use words like concept when talking about this at a high level.

[Xarlas]

Like, if you make a cartoon of Mickey Mouse fighting it out with Donald Duck, and they're clearly infringing depictions... you'll get sued. Stuff like my ships or portrait packs and so forth? No, because it's so far removed from any one work in particular. People wanting to be ethical merely have to use the tools to produce works in a reasonable, ethical way, for the most part, like my workflow demonstrated.

That is independent of AI use though. You will get sued if you draw it yourself too. Which is how it should be. If you use tools to create something that violates someone else's legal rights, you can get sued (or convicted if it was illegal) the same way if you would have done it yourself

[David]

There may be more, but here are a few public things trained on David's art.

https://civitai.com/models/52480
https://civitai.com/models/30
https://civitai.com/models/8296
https://huggingface.co/Severian-Void/Starsector-Portraits

Three are from Severian Void, one posted by a Kiktamo.

HuggingFace is like Github for models, Civitai is like, idk, NexusMods. Both are extremely searchable by the public and indexed by Google.

I'm not sure if David is aware of this or not - I know it's not my fight, but *specifically* copying one artist's style and making it easily usable by anybody (rather than something that you probably wouldn't find unless you were part of this modding community, ie not indexed by Google) seems different from training a general-purpose model from hundreds or thousands of artists. Or maybe it isn't, and the only difference is that these websites have options to file takedown requests. Maybe you saw these months ago and have already had this discussion.

As a baseline: If someone asked me for permission to do this in general, I would answer "no".

If someone did this with the assets I sell via Unity Asset Store, Cubebrush, Itch.io, I would consider the spectrum of polite request to legal means to remove the models, depending on context -- and I do explicitly request in the documentation for those assets (those which I've updated lately, anyway) that they not be used as input for procedural image generation.

For Starsector, the copyrighted material I create is owned by Fractal Softworks so, as a baseline, it is Alex's decision what to do with it.

There is such a thing as moral rights for a visual artist's works, though they are weakly enforced and not often respected. I'll quote a relevant section:

The right of attribution allows an author to enforce the attribution of their work, prevent the misattribution of their work to another author, and permits the author to retain anonymous or pseudo-anonymous ownership of the work. The right of integrity does its best to prevent distortion or modification of their work, easing an artists' worries surrounding negative defamation directly applied to their work affecting their own personal, creative, or professional reputation through misrepresentation.

The models linked above do credit me in the remote sense via crediting Starsector. The right of integrity is... dicey, but I/we've already decided to open that up via the policy of allowing kitbashing. And as for negative defamation through misrepresentation/association... well, that comes down to what people do with the images, which is a matter distinct from the tool used to generate them.

Due to practical considerations already noted, and the non-commercial nature of game modifications, I don't strongly object on ethical grounds to procedural generation of images for Starsector mods created using Starsector assets (see also: kitbashing). This does not speak, of course, to those copyrighted images and works of other artists used to populate the datasets of these procgen tools.

[Wispborne]

As a baseline: [...]

Understood, thank you for the response!

[xenoargh]

FWIW, I've just tried one of the LoRAs out, and have yet to have something pop out that was anything like as good as I'd have gotten from img2img plus a prompt.

The resemblance to David's work is... eh, let's say it's not very clear. However this was trained, it produced really samey results left to its own devices.


Left four is the portrait LoRA, just text prompts. Note that they all feel like clones. Center left is just me using one of the usual suspects, no img2img to guide things or postwork to fix framing. Right of center is David's work. Right four are both with and without the LoRA, but using two of the game's portraits as img2img base files. Note how they're suddenly very similar... but they still don't have the right feel. If I don't tell you which one's using the LoRA, I don't think you'd be able to guess.

They still don't look much like David's work to me. Maybe somebody better at this could get closer, or with a better img2img base emphasizing where to go.

A lot of the samey look in the first four is the low sample numbers; there are literally zero blonde men with beards wearing helmets in the SS portraits, which is why I picked that prompt out. The second set obviously has problems with framing; that might or might not be correctable w/ prompts, or simply running sheer numbers. But it's missing all of the warmth of David's work. The last four (again, the far-right set) are close on things like the eyes, but are missing a lot of other things. Could these be fixed in post to look like reasonable facsimiles? Probably, but this wasn't like doing photocopies, not by a long shot.

[Wispborne]

general ai chatter

(maybe we should have a separate thread for general AI chatter?)

[xenoargh]

I'm just exploring the practical ramifications of the LoRAs here.

Like, is this EZ Infringement, even with a low sample size, or not?

My vote thus far is "no", but I've only been using Stable Diffusion for a few months. With the ones on the far right, I could fix them to be... OK, after post.

But it isn't no-skill work; David's color grading and contrasts are wildly different, for example, and there's nothing like his brushwork in these. And ofc, it goes without saying... but to get this many "OK, I guess" attempts at copycats, I had to run SD quite a few times; this was after setting up maybe a dozen runs last night with different parameters. It would've taken me less time to attempt a copycat by hand.

Anyhow, no, I'm not going to put out a SS portrait pack using this, period. While I'm not convinced this is actually infringing, I'm also not in favor of "anything goes" for this kind of thing, largely because I think it'll eventually devolve into people fighting about it. I will not be training any LoRAs on anything remotely SS-related, and the ones I've been using are made in a way that's actually pretty ethical, so far as I can determine.

[RoquetheRogue]

They still don't look much like David's work to me. Maybe somebody better at this could get closer, or with a better img2img base emphasizing where to go.

Based on my own experimentation, you can get them to somewhat match the vanilla design by adding "conte, brush strokes and sabattier effects" in the keywords at the end.
as taught in the Severian-Void stable diffusion model, otherwise, it's simple to try and add these effects yourself using any graphic tool

Heh.. I have been using paint.net to fix some of the ai gen mistakes in the portraits in my mods
one major flaw being the file size that needs to be resized, even if these portraits extremely overdone, that semblance to vanilla and well, it's better than nothing.

This is just art gen AI, I want to see if someone has figured out a way to make officers talk like ChatGPT or CS, I have seen mods for bannerlord, skyrim and even Rimworld with integrated chatbots as in either a commentator or actual npcs having conversations about the world or their created character.

imagine if we somehow find a way to feed the AI faction lore from the game and from the mod and see their opinion on how modded factions and vanilla factions blend, or have ingame advisors helping with war scenarios if you need it.
Like, ask one of your chatbot officers/advisors about potential targets and they come up with this strategy to attack and take over the Thule system if using Nex.

Or, if they can be programmed to generate dynamic missions ingame, like quest-giver npcs, a pirate or smuggler type coming up with a idea for unique or skyrim type "radiant quests", genered based on a pool of information limited by their archetype, hmmm....

[Alex]

(maybe we should have a separate thread for general AI chatter?)

(Yeah, this thread is specifically regarding the policy for its use in mods, so let's please not have the thread go off-topic.)

[xenoargh]

Sorry. I thought that was relevant, as doing a LoRA with a mod's content now seems like a pretty dubious proposition from where I sit, unless it's very large and consistently styled. Nobody's going to clone BRDY's brilliant pixel-art tricks or MShadowy's flowing style with ease, I think. That's a bit of a relief, really.

I have seen mods for bannerlord, skyrim and even Rimworld with integrated chatbots as in either a commentator or actual npcs having conversations about the world or their created character.

That's really cool. I've written several things that hook into LLMs, and, with appropriate training, this kind of thing's quite practical. Using ChatGPT, not so much, mainly because it was a mod and it took off, you'd be on the hook for the Azure Cloud fees (it costs ~3X the cost of a web-search hit to query ChatGPT atm). But there are now several "wild" LLMs that can be integrated into software and run locally. Anyhow, I'll bow out of this discussion, as this is also kind of a derail.