Sunday 15 August 2021

AI Art -- VQGAN+CLIP

After reading this article in the New Yorker, I tried the AI (artificial intelligence) "text-to-image" program again -- VQGAN+CLIP -- to alter the drawings from my book:


Texts: "In the style of Egon Schiele"


Texts: "In the style of Moebius"

But is this really so different from what others get by referring to Moebius?



Texts: "In the style of Mucha"


Texts: "Giorgione Sleeping Venus"


Texts: "In the style of Otto Dix"


Texts: "In the style of OP art"


Not everything came out wonderful, I posted only the best results above.


PROGRAM

There are a few versions of the Google Colab notebook for VQGAN+CLIP (the third one works best for me, without hanging up so much):

  • NOTE:  The first step converts the uploaded image into a 512 x 512 dpi SQUARE image.

CHEATING 
"In the style of..." 
  • After I added my image to the "init_image:" field, I then began the entry at the top --  the "texts:" field --  with "In the style of ..."  Without that phrase the result can be polluted with things other than just the style of that artist.


  • For instance, when I just typed in the text "GIGER," H R Giger's face surfaced as much as his style.  I had to use another online AI program -- Nvida's "Image Inpainting" demo -- to eliminate the face and create a whole new coherent fake, but acceptable, posthumous H R Giger:

I edited out the Giger face (left)
online program


FAKE GIGER

VERIFIED ORIGINAL FAKE:  Below is the larger image of the new fake H R Giger created with VQGAN+CLIP and Nvidia "Image Inpainting."  The Russian search engine YANDEX gave me a lot better selection of similar images than Google, with their "reverse image search" -- but none of them were the same as this fake AI Giger I created below:

Successful FAKE H R GIGER
gives me a lot more accurate selection of similar images
but the exact image above does not come up!!!


VQGAN+CLIP seems to generate good fake Gigers, better than images in the styles of other famous artists

PIXEL DRAW

There is a VQGAN program -- CLIPIT PixelDraw --more specifically constructed to make pixel art (via Boing Boing):





SIMPLER INTERFACES

Previous "text-to-image" programs

KAPWING

KAPWING has a one step "text-to-art" VQGAN+CLIP process:




I typed in "Krrrl Drawings,"
and this is what the KAPWING program generated


I typed in "Art of the future,"
and this is what the KAPWING program generated



A HUGGING FACE
  • A HUGGING FACE alsohas a simpler interface for VQGAN+CLIP, but makes smaller images, and no videos:


Screen shot of A HUGGING FACE simpler interface
 when creating the image above
(click to enlarge)



The drawings were taken from my book -- FINISH MY FIGURE DRAWINGS:



  • #VQGAN -- I was inspired to alter my figure drawings by @unltd_dream_co  who put out a challenge to alter his drawings with this VQGAN+CLIP program.




I tried the VQGAN+CLIP program previously, and posted on July 13th:




COMPARING AI PROGRAMS

Note how the AI style transfer program Deep Dream Generator gives very different result (top row in image below) than this "text-to-image" AI approach of VQGAN+CLIP (bottom image):

Top:  Deep Dream Generator style transfer
Bottom: VQGAN+CLIP input "In the style of Egon Schiele"
 applied to my same drawing


DOUBLE AI
with Deep Dream Generator

I then used Deep Dream Generator to transfer the style of the VQGAN+CLIP result to the original drawing, and got a better end result (below image, far right):

transferred the style of the VQGAN+CLIP result
to the original drawing





I experimented again after FIRST running the VQGAN+CLIP program -- then NEXT by blending the first two iteration results in Deep Dream Generator.  I transferred the style of the second iteration (middle colored) to the first iteration (left, which is not in color) to get the Deep Dream result (on the right):



What would happen if I looped my drawing through a variety of different AI programs?  

Note that there is an online AI program that increases the resolution -- waifu2x.

I've altered my drawings before with other AI (artificial intelligence) programs:


Is there a way to get the VQGAN+CLIP program to use the latent space data bases I created based on my drawings (would this Colab notebook do it if one directed it to one of my pkl files?):

  • This Github by CompVis Heidelberg explains the data bases used in the VQGAN+CLIP programs (taming transformers)

No comments:

Post a Comment

Note: only a member of this blog may post a comment.