Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Severian Void]

Starsector Portr-Ai-t Pack and Ai model
Portrait Pack Download
Stable Diffusion Image Generation Model & LORAS
Civitai Link SD1.5 LORA version
NEW DEC 2024! Civitai Link SDXL LORA version

As someone whose art is not as good as I would like, but who has just enough technical capability to consider attempting this, I have completed my month long project. I took it upon myself to be the "anyone with a small bit of knowledge" person that went about training an image generation model to generate portraits in the vanilla style of starsector. I really want to see everyone able to include high quality custom portraits as part of their mods regardless of how capable they might be at producing them. Along with the model files I also edited and put together a vanilla style portrait pack which contains 500 out of the over 40,000 portraits that were generated during training.

The image here below contains 100 out of the total 500 portraits in the portrait pack

Raw Output Neural Network Samples

These are additional portraits right out of generation from the neural network, none of these portraits are actually in the portrait pack these are just additional samples so that people can get an idea of what the raw generations look like. These are portraits that were selected in the first pass while I was selecting out of the 40,000 portraits generated during training, but did not make it through the second pass of selection, so they are still of above average quality. Portraits are generated at 512x512 and for the portrait pack all I had to do was cut out the backgrounds to make them black and then downscale to 128x128. The portraits are usually generated with some kind of filled background but it will generate a solid black or white background about 20-30% of the time. Links to full size versions of these sample sheets with 512x512 portraits are available and people are free to use these portraits however they like.

Spoiler

Full size image with 512x512 portraits

[close]

Spoiler

Full size image with 512x512 portraits

[close]

Spoiler

Full size image with 512x512 portraits
These are portraits that generated with solid black backgrounds directly

[close]

[close]

Credits

Alex and David - For the absolutely amazing base game artwork
Anotoki - For testing the model and being the first person to provide feedback and results from running the model themselves
Lortus - For additional feedback on the portrait generation
Unofficial Starsector Discord - for waiting out the long delay of me actually getting this released

[close]

Changelog

Portrait Pack
Version 1.2: Sharpened portraits and fixed main preview collage using unedited generations
Version 1.1: Fixed Tri-Tachyon portraits using independent not corporate portraits (thanks Crilltic)
Version 1.0: initial release

Model Files
Version 4.0: New LORA trained using SDXL as the base model, has faction and style keywords, can be found in the 4_LORA folder on huggingface or at the new CivitAI link
Version 3.0: Trained LORA rather than a full model, use it with other SD1.5 models as a plugin rather than the old clunky full model, can be found on both the old link in the 4_LORA folder or the CivitAI link
Version 2.1: Tweaked version of 2.0, roughly generates the same image as 2.0 but usually with less defects
Version 2.0: Same training regime as 1.0 but based on SD 1.5
Version 1.0: Initial model trained on SD 1.4

[close]

[Lortus]

Congrats on the release!

[RoquetheRogue]

This is Really Good! 

[Beobachter]

Knowing how AI artwork is generated from an existing portfolio, is David (the artist) okay with this?

[Farya]

Knowing how AI artwork is generated from an existing portfolio, is David (the artist) okay with this?

I doubt there would be any problem for as long as these generated assets are not used commercially.

[Crilltic]

I think currently Tritach isn't set up to use the corporate images generated, they only use the independent ones.

[Severian Void]

I think currently Tritach isn't set up to use the corporate images generated, they only use the independent ones.

Yup, you are correct thanks for catching that I have updated the links with a fixed version so all the portraits should be used correctly now.

[Bastion.Systems]

I think these could do with some sharpening, I put these through Photoshop 250% strength smart sharpen and it really made these look much closer to base game style.
There are a few with scuffed eyes, asymmetrical floaty headpieces and strange visor reflections. Few, if you look closely, are just cursed variations of vanilla faces. You could easily throw out 40% of the pack, do some minor manual edits and up the quality by a lot.
I'll try and mess around with the model myself, as the overall quality is very impressive, there are some really inspired portraits in the set.

[Severian Void]

I think these could do with some sharpening, I put these through Photoshop 250% strength smart sharpen and it really made these look much closer to base game style.
There are a few with scuffed eyes, asymmetrical floaty headpieces and strange visor reflections. Few, if you look closely, are just cursed variations of vanilla faces. You could easily throw out 40% of the pack, do some minor manual edits and up the quality by a lot.
I'll try and mess around with the model myself, as the overall quality is very impressive, there are some really inspired portraits in the set.

Yeah, after over a month working on the training and looking at tens of thousands of generated portraits, I finally decided I just wanted to get it out there. None of the portraits in the pack were generated in any form of intentionally guided result and are all just images randomly generated as samples during the training process. I did still want to include a portrait pack along with the models so I did the absolute minimum of editing, just cutting out the generated backgrounds to black and in a few cases cleaning up some stray elements. I still have a few folders with thousands that I pulled aside for being particularly interesting but that require more editing to be truly good. The 1.5 version of the model which greatly improved face generation as well as the associated hypernet which pushes the quality even higher, only finished training 2-3 days before I released so most of the portraits in the pack were generated from the 1.4 version.

That said after reading your comment I did come to realize that my main preview of 100 portraits from the pack was using the unedited versions which is why they still had some floaty objects in a few of them. The actual edited portraits in the pack had them removed. I did like your suggestion though and ran everything through smart sharp and it does seem like an overall improvement. I am planning to make a few more smaller addon packs with maybe only 20-30 portraits at time for specific factions, that way I can spend a lot more time on each portrait individually to really push the quality high.

Let me know what your results are like messing around yourself, I am really curious to see what kinds of things other people come up with.

*Added edit* I actually just updated the models with a tweaked version 2.1 which generates roughly the same structured image as 2.0 but generally with less defects and better resolved faces. Still some issues, but overall I think its another improvement

Spoiler

Some generated examples comparing 2.0 vs 2.1 generations for the same seed

[close]

[Bastion.Systems]

Here is my first try with the 2.0 version of your hypernet using a portrait of Alexander II as the img2img base, I painted the background of the original image black in photoshop:

Spoiler

[close]

Generated 100 images, about 40 were good enough (a very high percentage!), did some minor fix up and then batched em' through photoshop auto resize and smart sharpen on 50%.
Second row, fourth one is my favorite, such sad melancholic look. Some really look like gruff Russian naval officers, I really appreciate the fabulous mustaches.
The img2img generation probably needs some keywords to improve the intensity of the lighting, it did generate all pictures with a perfect black background, so might be a good way to save time.
The coat is a little overly greebly, maybe my image size is set too high?
Here are the settings:

Spoiler

Russian officer with a mustache wearing a peaked cap, black background, starsectorportrait, sabattier effect
Steps: 40, Sampler: DPM2 a, CFG scale: 7, Size: 704x704, Denoising strength: 0.75, Mask blur: 4

[close]

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.

[Severian Void]

Hey those are awesome! If you are using img2img and the starting image has a black background, it will almost guarantee that all generated images also have black backgrounds. Otherwise with just txt2img its maybe 40-50% have either solid black or solid white and the rest get random stuff. I am a little surprised that its still so low for generating black backgrounds considering that solid black background is the single most common factor between every training image.

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.

Hah hah, I probably shouldn't have used that one as one of the example prompts since I had to try a lot of variations of prompts and generations on that before I finally got it to spit one out. I think the bigger issue is there is only a single image in the training set I trained with that has that style of hat, so it doesn't have a whole lot of training for it. Also doing a quick searching of the dataset that the base stable diffusion model was trained on, there are very few images as well that are matching balaclava, most of which are actually scenic photos of a city by that name, and a house project from some design firm in australia called Balaclava Road. So rather surprising I got one at all.

You can see here in a small database sample from the further training database that SD1.5 was refined on some of the images for "balaclava"
https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images?_search=balaclava

It gets even worse if you look at some examples of the clip retrieval from the larger database that made up the bulk of stable diffusions training
https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn5.laion.ai&index=laion5B&query=balaclava&useMclip=false

Its honestly a testament to just how incredible the generation algorithms are and that they even work, considering how bad the image dataset for training is tagged and sorted

[Vundaex]

Very good!
I prefer this over other portrait packs because of visual consistency.
Are you planning to add more?

[Farya]

Here is my first try with the 2.0 version of your hypernet using a portrait of Alexander II as the img2img base, I painted the background of the original image black in photoshop:

Spoiler

[close]

Generated 100 images, about 40 were good enough (a very high percentage!), did some minor fix up and then batched em' through photoshop auto resize and smart sharpen on 50%.
Second row, fourth one is my favorite, such sad melancholic look. Some really look like gruff Russian naval officers, I really appreciate the fabulous mustaches.
The img2img generation probably needs some keywords to improve the intensity of the lighting, it did generate all pictures with a perfect black background, so might be a good way to save time.
The coat is a little overly greebly, maybe my image size is set too high?
Here are the settings:

Spoiler

Russian officer with a mustache wearing a peaked cap, black background, starsectorportrait, sabattier effect
Steps: 40, Sampler: DPM2 a, CFG scale: 7, Size: 704x704, Denoising strength: 0.75, Mask blur: 4

[close]

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.

> Alexander II as Hegemony captain.
Tanc a lelek starts playing.

[Rotok]

This is amazing! Have you given any thought to doing this with the Illustrations? (the flavor art when you doc, or interact with things).
Would love to see more star port art:)

[banano of doom]

This is amazing! Have you given any thought to doing this with the Illustrations? (the flavor art when you doc, or interact with things).
Would love to see more star port art:)

https://fractalsoftworks.com/forum/index.php?topic=25649.0