This is the result of training with a mixed style/object dataset kindly provided by @Pashahlis at a learning rate of 1e-6 for 30 epochs (70 steps/epoch) with batch size 18.
validation
This model was trained using Victor C Hall's excellent Stable Diffusion finetuner EveryDream2. EveryDream2 configuration files for this training session are in this repo, here.
The configuration files enable a validation pass using a 15% split of the dataset with the noise seed held fixed during validation to give the following loss curve (stitched together from two runs of 60 epochs each):
Although the training ran for 120 epochs in total, the validation graph suggests that the best results are going to be at some point between epoch 10 and epoch 30:
This repository contains a diffusers format model for epoch 30:
It's available in InvokeAI by adding the diffusers repo id damian0815/pashahlis-val-test-1e-6-ep30
, or for manual download in .ckpt format if you're using a clumsier web UI: pashahlis-1e-6-ep30.ckpt.
... but is it finished training?
Training an SD model is subjective. Picking when to stop is a trade-off between an evaluation about how well the model reproduces the training data the way you want it to, vs how flexibly it is able to apply the new training data to novel outputs.
There are some generated image samples from each epoch to look at (generated with my python tool grate). For example, this one (warning: huuuge image, 20,000x10,000 pixels): . I'm satisfied that the training quality roughly follows the shape of the validation graph, but you might want to look at this image closely to verify for yourself that the best model is probably somewhere between epoch 30 and epoch 40.
- Notice how at epochs 30 and 40 the
ancient temple
prompt produces a variety of different temples with different seeds. By epoch 50 some weird artefacts are starting to creep in, with the results becoming progressively more monotonous and, especially beginning epoch 80, increasingly bizarre. - The
scottish ruined castle
images only start lookingruined
by epoch 40, but already at epoch 50 they are showing signs of rigidity, ignoring the difference in seed to produce the same style of castle each time. - The
fantasy orchard
prompt is notably resilient, but thevibrant fairy village
two columns over has lost all trace offairy village
already by epoch 40. - The
snail wedding ceremony
survives until epoch 70, but thebrushstrokes, canvas, fine art
quality of the final prompt has been replaced by an anime aesthetic by epoch 50.
try them yourself
If you want to try them out for yourself, other epochs are available at damian0815/pashahlis-val-test-1e-6-ep40, damian0815/pashahlis-val-test-1e-6-ep80, damian0815/pashahlis-val-test-1e-6-ep110,