AI Art -- VQGAN+CLIP

After reading this article in the New Yorker, I tried the AI (artificial intelligence) "text-to-image" program again -- VQGAN+CLIP -- to alter the drawings from my book:

Krrrl Drawings Playlist

KRRRL Drawings altered by 

VQGAN+CLIP

Drawing 025

Texts: "In the style of Egon Schiele"

Drawing 002

Texts: "In the style of Moebius"

But is this really so different from what others get by referring to Moebius?

Drawing 005

Texts: "In the style of Mucha"

Drawing 006

Texts: "Giorgione Sleeping Venus"

Drawing 015

Texts: "In the style of Otto Dix"

Drawing 034

Texts: "In the style of OP art"

Not everything came out wonderful, I posted only the best results above.

PROGRAM

There are a few versions of the Google Colab notebook for VQGAN+CLIP (the third one works best for me, without hanging up so much):

• VQGAN+CLIP (z+quantize method with augmentations)
• VQGAN+CLIP An PUBLIC COPY
• VQGAN+CLIP (with pooling)👍
• This is not DALL-E

• NOTE:  The first step converts the uploaded image into a 512 x 512 dpi SQUARE image.

CHEATING 

"In the style of..." 

• After I added my image to the "init_image:" field, I then began the entry at the top --  the "texts:" field --  with "In the style of ..."  Without that phrase the result can be polluted with things other than just the style of that artist.

Best word choice

• For instance, when I just typed in the text "GIGER," H R Giger's face surfaced as much as his style.  I had to use another online AI program -- Nvida's "Image Inpainting" demo -- to eliminate the face and create a whole new coherent fake, but acceptable, posthumous H R Giger:

Drawing 001

I edited out the Giger face (left)

with the Nvida AI "Image Inpainting"

online program

FAKE GIGER

VERIFIED ORIGINAL FAKE:  Below is the larger image of the new fake H R Giger created with VQGAN+CLIP and Nvidia "Image Inpainting."  The Russian search engine YANDEX gave me a lot better selection of similar images than Google, with their "reverse image search" -- but none of them were the same as this fake AI Giger I created below:

Drawing 001

Successful FAKE H R GIGER

YANDEX reverse image search

gives me a lot more accurate selection of similar images

than Google's reverse image search  --

but the exact image above does not come up!!!

VQGAN+CLIP seems to generate good fake Gigers, better than images in the styles of other famous artists

•  Other fake Gigers -- based on statues
• Was Giger a Qi Baishi fake?

PIXEL DRAW

There is a VQGAN program -- CLIPIT PixelDraw --more specifically constructed to make pixel art (via Boing Boing):

CLIPIT PixelDraw

after typing in "Krrrl Drawings"

SIMPLER INTERFACES

Previous "text-to-image" programs

• Computer Vision Explorer  -- I mentioned this "text-to-image" program at the end of my January 12, 2021 blog post.

• Generative Engine by Runway ML is another earlier "text-to-image" program online (from the more push button art blog post)

KAPWING

KAPWING has a one step "text-to-art" VQGAN+CLIP process:

Simply type text into the KAPWING prompt

and it will generate a picture

I typed in "Krrrl Drawings,"

and this is what the KAPWING program generated

I typed in "Art of the future,"

and this is what the KAPWING program generated

• Here is an article by an editor from Kapwing about how their interface is not being used to create art

A HUGGING FACE

• A HUGGING FACE alsohas a simpler interface for VQGAN+CLIP, but makes smaller images, and no videos:

My drawing altered in

 A HUGGING FACE

Screen shot of A HUGGING FACE simpler interface

 when creating the image above

(click to enlarge)

The drawings were taken from my book -- FINISH MY FIGURE DRAWINGS:

The altered drawings in this blog post

all came from my book

• #VQGAN -- I was inspired to alter my figure drawings by @unltd_dream_co  who put out a challenge to alter his drawings with this VQGAN+CLIP program.

• I was especially impressed by this VQGAN+CLIP artwork posted on Twitter by "ya" -- "Oasis city flythrough with spatial interpolation and 3D depth mapping."  It has a very Moebius feel.

Examples of some VQGAN+CLIP artworks

• VICE article: AI-Generated Art Scene Explodes as Hackers Create Groundbreaking New Tools
• We asked a new kind of AI art tool to make 'paintings' of Australia

I tried the VQGAN+CLIP program previously, and posted on July 13th:

My previous VQGAN+CLIP

And on July 6th -- Javelina on a unicycle.

---

COMPARING AI PROGRAMS

Note how the AI style transfer program Deep Dream Generator gives very different result (top row in image below) than this "text-to-image" AI approach of VQGAN+CLIP (bottom image):

Drawing 025

Top:  Deep Dream Generator style transfer

Bottom: VQGAN+CLIP input "In the style of Egon Schiele"

 applied to my same drawing

DOUBLE AI

with Deep Dream Generator

I then used Deep Dream Generator to transfer the style of the VQGAN+CLIP result to the original drawing, and got a better end result (below image, far right):

Deep Dream Generator

transferred the style of the VQGAN+CLIP result

to the original drawing

Edited Deep Dream Generator 

final result

I experimented again after FIRST running the VQGAN+CLIP program -- then NEXT by blending the first two iteration results in Deep Dream Generator.  I transferred the style of the second iteration (middle colored) to the first iteration (left, which is not in color) to get the Deep Dream result (on the right):

Drawing 002

Using Deep Dream Generator

to transfer the style of the second iteration

to the first iteration

What would happen if I looped my drawing through a variety of different AI programs?  

Note that there is an online AI program that increases the resolution -- waifu2x.

I've altered my drawings before with other AI (artificial intelligence) programs:

• Playform AI Experiments (2021)
• Aikphrasis AI exhibition, curated by Holly Grimm (2020)
• Artificial Intelligence creating Fake Bad Krrrl Drawings (2020)
• AI Art Review (2019)
• Holly AI (2018)
• AI Painting (my first blog entry on AI -- 2017)

Is there a way to get the VQGAN+CLIP program to use the latent space data bases I created based on my drawings (would this Colab notebook do it if one directed it to one of my pkl files?):

• "Book_model_final.pkl" (364 MB)
• "Long_Krrrl_Drawings__final.pkl" (364 MB)

• This Github by CompVis Heidelberg explains the data bases used in the VQGAN+CLIP programs (taming transformers)

• Vice: DeviantArt Is Now Using AI to Spot People Selling Stolen Art as NFTs
• Can I mint my AI altered drawings as NFTs using Tezos currency?
• Crowdsourcing database:  Neural.art