Steps to Create 3D Model from 2D Images Using TripsoSR AI

In the media industry, converting 2D images into 3D is highly relevant for several reasons:

1. Movies and Animation: The film industry often turns 2D concept art into 3D models to create animated features or visual effects. This allows for more dynamic and engaging storytelling with characters and environments that feel lifelike.

2. Virtual Reality (VR): Media companies are using VR to create immersive experiences. Converting 2D images into 3D models is essential for building these virtual spaces that users can explore.

3. Augmented Reality (AR): AR experiences often overlay 3D models onto the real world. Media companies can enhance print and digital content with AR to engage users in interactive ways.

4. Video Games: The gaming industry relies on 3D models to create the environments and characters that players interact with, providing a more engaging and realistic gaming experience.

5. Advertising and Marketing: 3D models are used in commercials and promotional materials to showcase products more effectively. They allow potential customers to visualize products in a way that flat images cannot.

6. Interactive Media: Interactive experiences, such as educational apps or museum installations, often require 3D models to engage users and convey information interactively.

By converting 2D images into 3D, the media industry can produce content that is more engaging, interactive, and provides audiences with a richer and more immersive experience. However, in the past, creating 3D models was a time consuming process. It would require a 3D artist to create the model using a 3D modeling software like 3D Studiomax, Maya, Blender and others. However, with Generative AI technology, it is possible to achieve the same task now in seconds.

In this blog post we will show how to render a 3D model from 2D images purely using Generative AI. As an example model, we will attempt to render an Iron Man action figure into 3D.  

Let us dive into the AI we’ll be using to render these 3D images.

TripoSR 

TripoSR is a cutting-edge, open-source AI model for rapid 3D reconstruction from single images, developed collaboratively by Tripo AI and Stability AI. 

‍

By implementing the principles of the Large Reconstruction Model (LRM), TripoSR achieves significant improvements in the speed and quality of 3D model generation. Remarkably, the model is capable of producing detailed 3D models in under half a second when run on an NVIDIA A100 GPU.

Comparative evaluations have demonstrated that TripoSR outperforms existing open-source solutions in both visual and metric-based assessments across various public datasets. Comprehensive information on the model's architecture, its training procedures, and comparative analyses are available in this technical report. 

Launching TripsoSR

Now, lets see how to achieve this. You would need a little bit of programming experience.

The first step is to clone the github repository, and installing requirements.

$ git clone https://github.com/VAST-AI-Research/TripoSR.git
$ cd TripsoS
$ pip install -r requirements.txt

We selected the following as our input 2D image. This Iron Man action figure was generated using Stable Diffusion XL:

Place the image into the examples folder, and then run this command

python run.py examples/Iron_Man.png –output-dir output/

You will see the following log

‍

The results will be saved in the output folder. It will be in the form of mesh.obj file.

Download the mesh.obj file, and then you can load it onto any 3D viewing software to visualize it. I used https://fetchcfd.com/3d-viewer. 

Click on the leftmost icon to load your mesh.obj.

Alternatively, you can also launch their Gradio interface —

python -m gradio_app.py

‍

Final Words

Here’s the Colab notebook for our little 3D experiment above: https://colab.research.google.com/drive/1xcgGf8Q1KZ7aYZWHrQMv2w_NYZ0wcPVY?usp=drive_link

You can tweak this notebook as per your own needs to generate 3D models. Please feel free to reach out to info@superteams.ai in case you need any guidance to manage your AI related workflows.

We hope you enjoyed reading this article!