Wednesday 24 June 2020

2D photo to 3D -- PIFuHD


PIFuHD


I used the PIFuHD demo software (on Google's Colaboratory) to convert my 2D drawing into a full 3D OBJ file:

The resulting 3D OBJ  file
spinning in front


Still image
from the above


NoteI first cropped out all of the background with remove.BG, and downloaded the PNG file with a transparent background.  This is what I uploaded to PIFuHD.



I followed the YouTube PIFuHD tutorial video to get through the process.  Although the interface is in Chinese, the narrative was in English, so it was not too impossible to navigate.




This is an AI (artificial intelligence) paper that was submitted by researchers from both Facebook and USC, and forwarded to me by Holly Grimm.


The whole project was presented as a larger paper, with examples and 3 other YouTube explanations:

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization




WORKS BEST with
 PHOTOS of REAL PEOPLE 

The software demo did work with my more realistic drawing above (and not with my less realistic drawing), but the result wasn't great.  PIFuHD works best with realistic frontal photos of humans, which is what it was designed for.

Therefore I downloaded a free PNG file of a woman (with transparent background), taken from the front -- and the resulting 3D image was a lot better.

Better 3D results,



TWITTER RESULTS

People are posting their results on Twitter:




Interesting results:


Perhaps someone could slightly alter the input 2D form in Photoshop, or even Deep Dream Generator, and get interesting results using PIFuHD.



BOCCIONI

I tried PIFuHD with a photo of a Umberto Boccioni sculpture-  Spiral Expansion of Muscles in Action.  Amazingly it went through the whole process, but the result was not pretty.

Rendered in Blender
The Boccioni 3D result
was in two incoherent pieces


I was able to download this
at the end of the PIFuHD process


***

PIFuHD reminds me of around 2012, when we were 3D scanning with the now discontinued 123D Catch software from Autodesk (using Photogrammetry) -- to 3D scan this big float, for instance -- giving us the same kind of artifact debris around and inside the figure.  




Likewise we used Scanect and ItSeez3D (like this example) during the Tucson Sculpture Festival 2015.

***

There are some other interesting online experiments in this direction:



No comments:

Post a Comment

Note: only a member of this blog may post a comment.