Monday 31 August 2020

Runway ML -- 3rd "Model" (based on long poses)

I uploaded 348 drawings -- of almost all of the long poses from my blog (mostly from 3 hour sessions) -- to Runway ML to create a 3rd "model" using StyleGAN2 (an artificial intelligence program posted on Github). 




The video below reveals the whole process.  The first mosaic section shows the whole training process, every 100 steps, which took about 3 hours.  The second section is a "latent space walk" between three points in the AI "model" space; each point being a generated drawing.  The last section are some of the better results, cherry-picked from 1095 automatically generated fake drawings which Runway ML delivered as a Zip file.



Perhaps the morphing in the video below, between 5 generated fake drawings, best illustrates what is going on with the artificial intelligence program StyleGAN2.  This is a "latent spacewalk" video that I downloaded from Runway ML.




The third time is a charm!  This model was "trained" on the second "model" of my short pose drawings, which was "trained" on the first "model" of the drawings from my book.  The final product was very self-referential and incestuous.  


RESULTS

The video below consists of 500 fake drawings, in the style of my long drawings, generated by Runway ML.  The images that follow have been cherry-picked to show off the best results.





This third model handled the faces a lot better, and even delivered some quasi-realistic fake drawings in my style.














Lots of the results looked less realistic than my drawings, but had a certain unexpected weird charm anyhow.











And a lot of results did not even have a figure in them, but still preserved the touch and flow of my hand.











DEEP DREAM GENERATOR

I continued to warp the fake drawings with another AI (artificial intelligence) program -- Deep Dream Generator -- which applies the style and color of one drawing, onto another.  It is a free online program from Google (requires registration).

The fake drawing at the beginning of this blog post,
colored and warped in Deep Dream Generator




Double Whammy --
style from the 5th drawing of the "weird set,"
applied to the 7th drawing from the "more realistic set"


***

QUESTIONS

  • If I "trained" these 348 drawings for more than 3000 steps, would it make any difference?  The final FID Score was 80.13, while I hear that one should aim lower, for 30 to 50 FID Score.

Final FID score of 80.13
after training

  • If I "trained" longer, would the results look more realistic; or at least, would they look even more like my original drawings?  
  • The faces came out way better than those in from the training on the shorter drawings, both in the first model and in the second model.  If I "trained" this third model more, would the faces come out even better?
  • Perhaps I should have "trained" these drawings less, using only 1500 steps.  Would the resulting drawings look more like a hybrid between the short and long poses in my styles?
  • What if I "trained" on a data set of someone else's drawings?  If I "half-trained," using only 1500 steps, would the results look like a hybrid of my drawings and those of someone else?  
  • I was thinking of augmenting my data set with drawings from Egon Schiele, whom I admire.  That is, mixing my drawings with those of Egon Schiele, to make one data base to upload. Could I get enough drawings by Egon Shiele to make a good enough model?  And if so, perhaps I could just "half-train" my drawings on the model based on Egon Schiele, to get a nice hybrid style.

  •  While the whole point of this exercise is to crank out new drawings in my style, I was hoping that perhaps this AI program could even improve my drawings in the process.

No comments:

Post a Comment

Note: only a member of this blog may post a comment.