I built this out of my personal interst in 2018 in my free time as a senior CS undergraduate.
This program is able to reconstruct a 3D face model from a single front-face image with the help of C++, QT, OpenGL, OpenCV, eos/dlib libraries and surrey face model. Also, it automatically reads the .obj file with its texture and display the full model in front of you right after the reconstruction process!
The interactive UI was implemented using C++ Qt. It deserves some refinements when I have time.
Following is the model viewer UI, as you can adjust the model angles with sliders or drag with your mouse to adjust the pose of the model generated!
I used Visual Studio 2018 on Windows to do my project.
- In order to compile the program, you need to configure the following dependencies in order:
OpenCV -> OpenGL -> dlib -> eos -> Qt
- referenced Libraries:
https://github.com/patrikhuber/eos
I used a shape-only version of the Surrey Morphable Face Model as the skeleton of the model.
The full model is available at http://www.cvssp.org/facemodel.
Here is the face landmark detection part. By selecting 68 landmarks, a face could be identified and located in the image.
Given the input sample photo:
By running face detection module, my program identified the front face and its outline and some interest points, e.g. nose and eyes.
By implementing the texture extraction API, combined with Dlib face landmark detection data, the texture can be extracted from the original image.
The face texture could be presented in this sturcture:
Here comes the most challanging part.
- First, the program fits the original model to different face image, adjusts its surface parameters in order to present more natural and realistic outputs.
- Secondly, given an extracted texture of the face and the surrey face model, the program calculates a mapping between the coordinates from a 2D image to a 3D model.
Some final outputs:
