Fractal Softworks Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Starsector 0.97a is out! (02/02/24); New blog post: New music for Galatia Academy (06/12/24)

Pages: [1] 2

Author Topic: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model  (Read 12328 times)

Severian Void

  • Ensign
  • *
  • Posts: 8
    • View Profile
[0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« on: October 26, 2022, 12:20:25 AM »


As someone whose art is not as good as I would like, but who has just enough technical capability to consider attempting this, I have completed my month long project. I took it upon myself to be the "anyone with a small bit of knowledge" person that went about training an image generation model to generate portraits in the vanilla style of starsector. I really want to see everyone able to include high quality custom portraits as part of their mods regardless of how capable they might be at producing them. Along with the model files I also edited and put together a vanilla style portrait pack which contains 500 out of the over 40,000 portraits that were generated during training.

The image here below contains 100 out of the total 500 portraits in the portrait pack


Raw Output Neural Network Samples
These are additional portraits right out of generation from the neural network, none of these portraits are actually in the portrait pack these are just additional samples so that people can get an idea of what the raw generations look like. These are portraits that were selected in the first pass while I was selecting out of the 40,000 portraits generated during training, but did not make it through the second pass of selection, so they are still of above average quality. Portraits are generated at 512x512 and for the portrait pack all I had to do was cut out the backgrounds to make them black and then downscale to 128x128. The portraits are usually generated with some kind of filled background but it will generate a solid black or white background about 20-30% of the time. Links to full size versions of these sample sheets with 512x512 portraits are available and people are free to use these portraits however they like.

Spoiler
Full size image with 512x512 portraits
These are portraits that generated with solid black backgrounds directly
[close]
[close]

Credits
Alex and David - For the absolutely amazing base game artwork
Anotoki - For testing the model and being the first person to provide feedback and results from running the model themselves
Lortus - For additional feedback on the portrait generation
Unofficial Starsector Discord - for waiting out the long delay of me actually getting this released
[close]
Changelog
Portrait Pack
Version 1.2: Sharpened portraits and fixed main preview collage using unedited generations
Version 1.1: Fixed Tri-Tachyon portraits using independent not corporate portraits (thanks Crilltic)
Version 1.0: initial release

Model Files
Version 3.0: New training of a LORA rather than a full model, use it with other trained models as a plugin rather than the old clunky full size model, can be found on both the old link in the 4_LORA folder or the new link
Version 2.1: Tweaked version of 2.0, roughly generates the same image as 2.0 but usually with less defects
Version 2.0: Same training regime as 1.0 but based on SD 1.5
Version 1.0: Initial model trained on SD 1.4
[close]
« Last Edit: April 27, 2023, 05:19:07 PM by Severian Void »
Logged

Lortus

  • Commander
  • ***
  • Posts: 109
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #1 on: October 26, 2022, 12:22:24 AM »

Congrats on the release!
Logged

RoquetheRogue

  • Commander
  • ***
  • Posts: 110
  • personal text.
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #2 on: October 28, 2022, 05:21:10 AM »

This is Really Good!  ;D
Logged
A mind that opens itself to a new idea never returns to it's original size.

Beobachter

  • Ensign
  • *
  • Posts: 46
  • Balancing? What is that?
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #3 on: October 28, 2022, 09:53:52 AM »

Knowing how AI artwork is generated from an existing portfolio, is David (the artist) okay with this?
Logged
Putting together a mod sometime in the distant future.

Farya

  • Commander
  • ***
  • Posts: 246
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #4 on: October 28, 2022, 12:18:12 PM »

Knowing how AI artwork is generated from an existing portfolio, is David (the artist) okay with this?
I doubt there would be any problem for as long as these generated assets are not used commercially.
Logged

Crilltic

  • Ensign
  • *
  • Posts: 13
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #5 on: November 01, 2022, 04:07:42 AM »

I think currently Tritach isn't set up to use the corporate images generated, they only use the independent ones.
Logged

Severian Void

  • Ensign
  • *
  • Posts: 8
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #6 on: November 01, 2022, 02:38:18 PM »

I think currently Tritach isn't set up to use the corporate images generated, they only use the independent ones.

Yup, you are correct thanks for catching that I have updated the links with a fixed version so all the portraits should be used correctly now.
Logged

Bastion.Systems

  • Captain
  • ****
  • Posts: 423
  • Special Circumstances LCU
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #7 on: November 05, 2022, 02:35:24 PM »

I think these could do with some sharpening, I put these through Photoshop 250% strength smart sharpen and it really made these look much closer to base game style.
There are a few with scuffed eyes, asymmetrical floaty headpieces and strange visor reflections. Few, if you look closely, are just cursed variations of vanilla faces. You could easily throw out 40% of the pack, do some minor manual edits and up the quality by a lot.
I'll try and mess around with the model myself, as the overall quality is very impressive, there are some really inspired portraits in the set.
« Last Edit: November 05, 2022, 02:54:34 PM by Bastion.Systems »
Logged

Severian Void

  • Ensign
  • *
  • Posts: 8
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #8 on: November 05, 2022, 07:31:22 PM »

I think these could do with some sharpening, I put these through Photoshop 250% strength smart sharpen and it really made these look much closer to base game style.
There are a few with scuffed eyes, asymmetrical floaty headpieces and strange visor reflections. Few, if you look closely, are just cursed variations of vanilla faces. You could easily throw out 40% of the pack, do some minor manual edits and up the quality by a lot.
I'll try and mess around with the model myself, as the overall quality is very impressive, there are some really inspired portraits in the set.

Yeah, after over a month working on the training and looking at tens of thousands of generated portraits, I finally decided I just wanted to get it out there. None of the portraits in the pack were generated in any form of intentionally guided result and are all just images randomly generated as samples during the training process. I did still want to include a portrait pack along with the models so I did the absolute minimum of editing, just cutting out the generated backgrounds to black and in a few cases cleaning up some stray elements. I still have a few folders with thousands that I pulled aside for being particularly interesting but that require more editing to be truly good. The 1.5 version of the model which greatly improved face generation as well as the associated hypernet which pushes the quality even higher, only finished training 2-3 days before I released so most of the portraits in the pack were generated from the 1.4 version.

That said after reading your comment I did come to realize that my main preview of 100 portraits from the pack was using the unedited versions which is why they still had some floaty objects in a few of them. The actual edited portraits in the pack had them removed. I did like your suggestion though and ran everything through smart sharp and it does seem like an overall improvement. I am planning to make a few more smaller addon packs with maybe only 20-30 portraits at time for specific factions, that way I can spend a lot more time on each portrait individually to really push the quality high.

Let me know what your results are like messing around yourself, I am really curious to see what kinds of things other people come up with.

*Added edit* I actually just updated the models with a tweaked version 2.1 which generates roughly the same structured image as 2.0 but generally with less defects and better resolved faces. Still some issues, but overall I think its another improvement
Spoiler
Some generated examples comparing 2.0 vs 2.1 generations for the same seed
[close]
« Last Edit: November 06, 2022, 02:10:46 AM by Severian Void »
Logged

Bastion.Systems

  • Captain
  • ****
  • Posts: 423
  • Special Circumstances LCU
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #9 on: November 06, 2022, 08:43:11 AM »

Here is my first try with the 2.0 version of your hypernet using a portrait of Alexander II as the img2img base, I painted the background of the original image black in photoshop:
Spoiler
[close]
Generated 100 images, about 40 were good enough (a very high percentage!), did some minor fix up and then batched em' through photoshop auto resize and smart sharpen on 50%.
Second row, fourth one is my favorite, such sad melancholic look. Some really look like gruff Russian naval officers, I really appreciate the fabulous mustaches.
The img2img generation probably needs some keywords to improve the intensity of the lighting, it did generate all pictures with a perfect black background, so might be a good way to save time.
The coat is a little overly greebly, maybe my image size is set too high?
Here are the settings:
Spoiler
Russian officer with a mustache wearing a peaked cap, black background, starsectorportrait, sabattier effect
Steps: 40, Sampler: DPM2 a, CFG scale: 7, Size: 704x704, Denoising strength: 0.75, Mask blur: 4
[close]

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.
« Last Edit: November 06, 2022, 02:05:18 PM by Bastion.Systems »
Logged

Severian Void

  • Ensign
  • *
  • Posts: 8
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #10 on: November 07, 2022, 02:11:26 PM »

Hey those are awesome! If you are using img2img and the starting image has a black background, it will almost guarantee that all generated images also have black backgrounds. Otherwise with just txt2img its maybe 40-50% have either solid black or solid white and the rest get random stuff. I am a little surprised that its still so low for generating black backgrounds considering that solid black background is the single most common factor between every training image.

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.

Hah hah, I probably shouldn't have used that one as one of the example prompts since I had to try a lot of variations of prompts and generations on that before I finally got it to spit one out. I think the bigger issue is there is only a single image in the training set I trained with that has that style of hat, so it doesn't have a whole lot of training for it. Also doing a quick searching of the dataset that the base stable diffusion model was trained on, there are very few images as well that are matching balaclava, most of which are actually scenic photos of a city by that name, and a house project from some design firm in australia called Balaclava Road. So rather surprising I got one at all.

You can see here in a small database sample from the further training database that SD1.5 was refined on some of the images for "balaclava"
https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images?_search=balaclava

It gets even worse if you look at some examples of the clip retrieval from the larger database that made up the bulk of stable diffusions training
https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn5.laion.ai&index=laion5B&query=balaclava&useMclip=false

Its honestly a testament to just how incredible the generation algorithms are and that they even work, considering how bad the image dataset for training is tagged and sorted
« Last Edit: November 07, 2022, 03:10:11 PM by Severian Void »
Logged

Vundaex

  • Commander
  • ***
  • Posts: 148
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #11 on: November 27, 2022, 10:55:30 AM »

Very good!
I prefer this over other portrait packs because of visual consistency.
Are you planning to add more?
Logged

Farya

  • Commander
  • ***
  • Posts: 246
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #12 on: December 01, 2022, 05:23:15 AM »

Here is my first try with the 2.0 version of your hypernet using a portrait of Alexander II as the img2img base, I painted the background of the original image black in photoshop:
Spoiler
[close]
Generated 100 images, about 40 were good enough (a very high percentage!), did some minor fix up and then batched em' through photoshop auto resize and smart sharpen on 50%.
Second row, fourth one is my favorite, such sad melancholic look. Some really look like gruff Russian naval officers, I really appreciate the fabulous mustaches.
The img2img generation probably needs some keywords to improve the intensity of the lighting, it did generate all pictures with a perfect black background, so might be a good way to save time.
The coat is a little overly greebly, maybe my image size is set too high?
Here are the settings:
Spoiler
Russian officer with a mustache wearing a peaked cap, black background, starsectorportrait, sabattier effect
Steps: 40, Sampler: DPM2 a, CFG scale: 7, Size: 704x704, Denoising strength: 0.75, Mask blur: 4
[close]

I first tried your sample prompt from readme and I think your current version might be tad over trained as it really did not want to give me a balaclava.
> Alexander II as Hegemony captain.
Tanc a lelek starts playing.
Logged

Rotok

  • Ensign
  • *
  • Posts: 13
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #13 on: December 01, 2022, 04:16:07 PM »

This is amazing! Have you given any thought to doing this with the Illustrations? (the flavor art when you doc, or interact with things).
Would love to see more star port art:)
Logged

banano of doom

  • Commander
  • ***
  • Posts: 240
    • View Profile
Re: [0.9.51a-RC6] Starsector Portr-Ai-t Pack and Ai Model
« Reply #14 on: December 01, 2022, 04:36:09 PM »

This is amazing! Have you given any thought to doing this with the Illustrations? (the flavor art when you doc, or interact with things).
Would love to see more star port art:)
https://fractalsoftworks.com/forum/index.php?topic=25649.0
Logged
Any and ALL sprites i ever posted on this forum are FREE to use. even if i'm using them myself. Don't ever, EVER ask for permission, or i will come to your home and EAT YOUR DOG!!!
i do NOT want to see my name appear in the credits section of any published mod and will consider it a personal insult.
Pages: [1] 2