While I have experience with 3D modeling, this was my first time building a model from an existing photograph, rather than shapes. I outline my process, challenges, and reflections in the following sections.

Taking photos

The first step was capturing the photos of my model. From practicing in class, we decided that it would be best—for both aesthetics and convenience—to capture the subject sitting on a backless chair. in this way, we would not have to differentiate the back of the chair from the rest of the body, which was only supposed to be from the torso up. We wanted to do our best to capture the entire body as a floating entity, not a body sitting on an object, so we found a stool with a skinny stem for this. Both of us sat with our leggs crossed and our hands folded in front of us. In total, I captured 180 images including the mask, which consisted of eye-level, overhead, lower-level, and close-up.

Masking

Creating a mask for each photo was the most challenging and time consuming part. After importing the mask, I found that the mask did not detect my model’s entire body in any of the photos. About 50% of the photos had the body partially detected. Fortunately, the mask was great at detecting the hair, which I anticipated would be far more difficult. I think this was also due to the fact that my model’s upper body was against the white background of the gallery wall. The lower legs, especially the shoes/shoelaces, was never detected by the mask. It was also often confused with objects in the background such as the polka dot balloons and the potted plant.

Building the mask for each photo was a learning process, as I wasn’t sure how accurate it needed to be, and there was not much advice online. Some artists recommended making it extremely precise, while others said that the cleaning of points in the later stage would make the masking quality negligible. To be safe, I opted to utilize the Scissor tool to cut out my model as precicely as I couldfor each of the 180 images.

Point Cloud

After applying the masks, I generated the point cloud and readjusted the region.

 

Dense Point Cloud – uncleaned

I think the masking and number/quality of photos taken must have been pretty accurate, as I did not need to do much deletion of points after getting the initial dense point cloud. I mainly focused on cleaning the shoes and pants, as the model seemed to have confused the shadow the body onto the chair with being part of itself.

 

 

Dense Point Cloud – cleaned

Cleaning the dense point cloud produced the following results.

 

Mesh + Texture

In the final steps, I was able to gauge how well my photographs were able to generate an accurate model. I think for the most part, my photos were quite successful. The upper body, hair, arms, and legs were extremely accurate and detailed. The woven sweater that my model was wearing was also nicely textured in the final result.

  

 

adding texture…

 

Model vs. Photo

Reflections + Challenges

I had a lot of fun with this project seeing each of the stages come together to make a unified 3D model. Now with my knowledge of traditional 3D modeling, I am eager to combine real life and computer generated objects into Unity.

Still, this project was rather time consuming and involved a lot of learning along the way. For one, I found that no matter how much I adjusted the settings, the masking feature of Metashape was very poor, and I ended up having to nearly manually mask every single photo. My partner managed to find a way to do this part in photoshop, so I hope to experiment with that for the next project.

I also thought that the pose we had chosen was a bit difficult. Because we were sitting on chairs, there was not a clear sense of where the model should start and where it should end. Do we include the chair? If so, how far does the camera need to be in order for us to capture all angles while maintaining detail. Should we not include the chair? If so, how do we mask it efficiently in Metashape, and how difficult will it be to create a model with so many different limbs sticking out? How do we take pictures of the bottom of the body if we are sitting? These were all questions that we were figuring out as we went, and eventually we decided on a pose and furniture to optimize how much of the ‘bottom’ we would be able to get, without being able to capture the ‘bottom’. In the future, we speculated what we could achieve if we had our models posing on some transparent surface.