Fractal Softworks Forum

Starsector => General Discussion => Topic started by: pixel_rice_bowl on October 02, 2022, 04:37:17 PM

Title: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: pixel_rice_bowl on October 02, 2022, 04:37:17 PM
This is a collection of 109 portraits that tries--as best as I could--to stay close to the vanilla lore and art style. The quality varies such that I could probably pare this down to 70 or 80 portraits, but here I figure it's better to include more than less.

The collection took about 40-50 hours to make. It was created using a combination of Stable Diffusion plus manual cleanup. I generated around 4,000 portraits across a couple dozen prompts using varying sampling models and settings and then painted and edited them in Krita and Aseprite. Since Stable Diffusion often creates portraits with the anatomical equivalent of severe dmods, most of my time was spent drawing over or editing results from decent enough bases as well as adding or fixing portrait and rim lighting and errant pixels. Removing outliers, I want to say the average portrait took about 30 minutes to an hour.

Edit: Here's a picture of the full sprite sheet:

(https://i.imgur.com/SbF399K.png)

I'm releasing these to be used by modders for any packs or mods. You can find a collection of all portraits as individual files here:
here (https://drive.google.com/drive/folders/10ObWcIRj3SrSP9uMsh0w8q3j_vneUPR5) (edit: or as a zip here (https://drive.google.com/file/d/1lktSVGPPZZRqEDyF0zzpk8J2N1Df-dZW/view?usp=sharing)).

Someone is welcome to put them into pack and post them to the forums on my behalf if so moved. You can use, edit, or otherwise modify these in anyway you like. No attribution or credit is needed.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Thaago on October 02, 2022, 08:09:30 PM
Welcome to the forum!

Wow, these are pretty great. Its interesting to hear the amount of post processing needed on autogenerated portraits to make them good. Did you have to feed the program the existing portraits or anything as training data, or was it just text prompts?
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Kos135 on October 02, 2022, 08:28:12 PM
Looks good! Nice to see a portrait pack that isn't apart from the vanilla style, or even worse, anime.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: pixel_rice_bowl on October 02, 2022, 08:41:22 PM
Quote from: Thaago on October 02, 2022, 08:09:30 PM
Welcome to the forum!

Wow, these are pretty great. Its interesting to hear the amount of post processing needed on autogenerated portraits to make them good. Did you have to feed the program the existing portraits or anything as training data, or was it just text prompts?

Thanks for the welcome! Starsector is one of my favorite games of all time and I've lurked here for a while.

It was a combination of text prompts and image feeding. The process fed into itself--once I found some good prompts, I used the resulting images to create new and similar images. Sometimes I used existing portraits, but with the way img2img works, it requires some pretty careful slider control to get something useful. (Last I read, training the models on specific image sets requires an industrial grade GPU--something closer to an A100 than a 3090--which isn't something I have access to.)

And, yeah, it usually requires a good amount of work to make a result look decent. I have a few years of pixel art and 3d rendering experience, which made the process smoother, but it's clear even with a lot of prompt magic that the vanilla portrait style won't be replaced by an AI. Many of the portraits are close in some regard but they don't have the same faction feel as vanilla and the overall quality is sometimes inconsistent (as I've followed reaction to the portraits elsewhere, it's clear I made a few basic errors on at least a few of them that a trained artist wouldn't have). These deficiencies are one reason I decided not to make them into an actual mod--it needs more quality control.

I'm sharing this set primarily to show others how much labor can be saved by generating concepts or workable bases. I hope skilled artists make use of these new tools to create some killer portrait packs.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: cytokine on October 03, 2022, 12:04:02 AM
Wow, these are incredible!
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: CrashToDesktop on October 03, 2022, 03:08:50 AM
Well done! I do appreciate having vanilla-style portraits, I think I'll be adding many of these to my runs from here on out.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Rusty Edge on October 03, 2022, 06:23:11 AM
X-8, Y-5 has the haunted look of a Vangaurd crewman who has seen a battle or two.
X-8, Y-9 looks like an older Bobby Draper.
X-5, Y-9 A proper prince charming.
X-11, Y-5 I don't know why, I just love this guy.

I'm recognizing quite a few of these faces. Definitely an impressive collection, quite a lot of personality and possible stories here.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: TheLemu on October 03, 2022, 06:35:41 AM
Finally something to replace those pesky anime/meme portraits mod authors push down everyone's throat lmao
Thanks for compiling these
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Brainwright on October 03, 2022, 08:15:12 AM
It's a good demonstration, and I can see the judgement calls you made to improve various individual portraits.

However, I wouldn't use these.  The first thing that struck me when I saw them as a whole is that they look the same as the vanilla portrait set.  Same general shape and the same color scheme.

When I want to add a portrait pack, it's to expand the general range of shapes and colors to generate unique impressions.  Viewed individually, the AI generated images might seem to have variety, but it's very shallow.  You cease to be able to differentiate between individual portraits if you use them in-game.

That's been my impression of AI-generated art so far.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: cytokine on October 03, 2022, 09:18:48 AM
Quote from: Brainwright on October 03, 2022, 08:15:12 AM
The first thing that struck me when I saw them as a whole is that they look the same as the vanilla portrait set.  Same general shape and the same color scheme. When I want to add a portrait pack, it's to expand the general range of shapes and colors to generate unique impressions.  Viewed individually, the AI generated images might seem to have variety, but it's very shallow.  You cease to be able to differentiate between individual portraits if you use them in-game.

Ideally I'd want a distinctive look for my own officers, but for other NPCs, what the game needs less of is the 4th wall-breaking moments where you run into a clone of say, your own administrator. For me, that was the main downside of playing with vanilla portraits only. So you need more portraits to minimize duplicates. But if you do run into a duplicate, some blandness will cover it up, while distinctness makes it blatant: you would remember seeing exactly that face somewhere else, just because it was memorable. So quantity > generic > distinct, IMO. Against that kind of immersion-breaking, at least.

Not saying this portrait pack is bland. They just fit right in.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Thaago on October 03, 2022, 10:06:07 AM
I took another look and one thing I noticed is that I think the AI has a bit of difficulty in the size of helmets/hats: some of them are crazy big!
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Brainwright on October 03, 2022, 10:44:09 AM
Quote from: cytokine on October 03, 2022, 09:18:48 AM
Not saying this portrait pack is bland. They just fit right in.

Not saying it's bland, it's that if you can keep most of these portraits in your memory, they start to blend together.  Your response to them is all the same.  It's the sort of thing a magician would use in a trick.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: cytokine on October 03, 2022, 11:35:48 AM
Quote from: Brainwright on October 03, 2022, 10:44:09 AM
It's the sort of thing a magician would use in a trick.
Well... perceiving the in-game world as vast and as real and as full of people is an illusion, and creating immersion means making that illusion believable. That's the goal. It's like, tricks all the way down, dude! Anyhow, "different strokes" and all that...
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Great Wound on October 03, 2022, 12:54:17 PM
Quote from: pixel_rice_bowl on October 02, 2022, 04:37:17 PM
Someone is welcome to put them into pack and post them to the forums on my behalf if so moved.

Done! (https://fractalsoftworks.com/forum/index.php?topic=25431.0)
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: cytokine on October 03, 2022, 01:58:37 PM
Quote from: Great Wound on October 03, 2022, 12:54:17 PM
Done! (https://fractalsoftworks.com/forum/index.php?topic=25431.0)
Oops, no faction file for the luddic path? I guess it would be identical to the one for the luddic church?

Quote from: pixel_rice_bowl on October 02, 2022, 04:37:17 PM
Since Stable Diffusion often creates portraits with the anatomical equivalent of severe dmods, most of my time was spent drawing over or editing results from decent enough bases
What are you talking about? I had a go at making starsector portraits with stable diffusion about a month ago, and those turned out perfect. Someone should make another portrait pack...
Spoiler
(https://i.imgur.com/3LGhrgN.jpg)
[close]
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Doctorhealsgood on October 04, 2022, 03:19:40 AM
An inspection fleet has been sent to your position.
Jokes aside this is pretty neat.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: BigBrainEnergy on October 04, 2022, 12:29:30 PM
Quote from: cytokine on October 03, 2022, 01:58:37 PM
Spoiler
(https://i.imgur.com/3LGhrgN.jpg)
[close]
Ah yes, here we can observe the Game Gerf in its natural habitat.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Severian Void on October 12, 2022, 08:35:27 PM
Quote from: pixel_rice_bowl on October 02, 2022, 08:41:22 PM
(Last I read, training the models on specific image sets requires an industrial grade GPU--something closer to an A100 than a 3090--which isn't something I have access to.)

Would love to know more about what kind of prompts you worked out to get the style so closely, I really struggled with managing that so instead I have been spending my last few weeks working on training models to use for my starsector mod making since I couldn't ever seem to manage decent results with just prompts alone.
I was planning to work on a portrait pack myself to go along with releasing the portrait model I have been training, but looks like a bunch of you have all beat me to it for AI generated portrait packs. Though I have been spending all my time trying to improve and train a better and better model so I haven't really stopped to make use of it yet even though the results are probably more than good enough.

I will make a full forum post when I finish up and release the model, with some tips and guides on things I have learned while using it.

This is a batch of images directly out of the custom model I have trained, I have thousands more of similar quality just sitting around now.
(https://i.imgur.com/9ZIlYBs.png)

The model is able to adapt quite well and will maintain the style even if its generating things well outside the scope of the training set. I posted a bunch more under the spoiler with some outlandish things that don't exist. Obviously some images still have issues not everything could be used directly out of generation but so much of it is just a few steps from usable.
Spoiler
The only editing I did to these was chop out the backdrops to make them black
(https://i.imgur.com/IwMYTSD.png)
Some of this stuff is not supposed to exist you have been warned!
Spoiler
These I haven't edited at all and are directly from generation, you can see that sometimes it gets the black background correct but frequently it fills it in, but overall it holds the style very well even on subjects well outside any of the training data.
(https://i.imgur.com/lnDiiEk.png)
[close]
[close]

If anyone wants any more info just contact me I am the same name on the unofficial starsector discord
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: CapnHector on October 13, 2022, 05:59:09 AM
Stable Diffusion doesn't run on my computer. I guess because it's an old laptop.

But what happens if you use it to generate new ships for Starsector based on the existing sprites?
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Severian Void on October 13, 2022, 12:21:36 PM
Quote from: CapnHector on October 13, 2022, 05:59:09 AM
Stable Diffusion doesn't run on my computer. I guess because it's an old laptop.

But what happens if you use it to generate new ships for Starsector based on the existing sprites?

So far the answer has been its very rough, training picks up the overall structure of the images very well even in the case of ships. The problem though is that the neural nets have several more or less compression steps its not really compression but rather changing the shape of the image but the end result is the same and you lose the high frequency details in the image. If you have ever seen what jpg compression does to detailed pixel art its a similar issue.

Additionally part of the reason why it can be so good at portraits is because the base network that the additional training was applied to was trained on hundreds of millions of pictures of people for tens of thousands of computational hours, but would not have been trained on nearly as many images of spaceships. So it has a much better understanding of the structure of a human and can recreate it much better just in general, so getting good results after modifying and applying additional training on something it knows well is a lot easier.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Havoc on November 18, 2022, 07:22:48 AM
Quote from: Great Wound on October 03, 2022, 12:54:17 PM
Quote from: pixel_rice_bowl on October 02, 2022, 04:37:17 PM
Someone is welcome to put them into pack and post them to the forums on my behalf if so moved.

Done! (https://fractalsoftworks.com/forum/index.php?topic=25431.0)

hi,
how do I install it?
it is not on my "Mods..." screen

btw. really nice pics and fits to the game
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: FastestDraw on November 18, 2022, 09:42:25 AM
Quote from: Havoc on November 18, 2022, 07:22:48 AM
Quote from: Great Wound on October 03, 2022, 12:54:17 PM
Quote from: pixel_rice_bowl on October 02, 2022, 04:37:17 PM
Someone is welcome to put them into pack and post them to the forums on my behalf if so moved.

Done! (https://fractalsoftworks.com/forum/index.php?topic=25431.0)

hi,
how do I install it?
it is not on my "Mods..." screen

btw. really nice pics and fits to the game

It should be just drag and drop the folder inside the downloaded folder to the mods folder. It doesn't handle nested folders great, and the download folder and mod folder have the same name, so its easy to mix up.
Title: Re: a modest portrait pack -- 100+ vanilla-adjacent portraits
Post by: Havoc on November 20, 2022, 11:01:41 AM
it does not occur in my mods list
also tried to add "AMPP" manually to the json

it was extracted to mods
tried also without nested subfolders

edit:
maybe I found the problem....several starsector installations and mod was in the wrong one :D

[attachment deleted by admin]