3D facial modeling with geometric wrinkles from images

dc.contributor.advisorDeng, Zhigang
dc.contributor.committeeMemberPavlidis, Ioannis T.
dc.contributor.committeeMemberChen, Guoning
dc.contributor.committeeMemberMayerich, David
dc.creatorDeng, Qixin
dc.date.createdMay 2023
dc.description.abstractRealistic 3D facial modeling and reconstruction have been increasingly used in many graphics, animation, and virtual reality applications. Currently many existing face models are not able to present rich details while deforming, which means lack of wrinkles while face shows different expressions. Also, to create a realistic face model for an individual is also needs complex setup and sophisticated works from experienced artists. The goal of this dissertation is to achieve an end-to-end system to augment coarse-scale 3D face models, and to reconstruct realistic face from in-the-wild images. I propose an end-to-end method to automatically augment coarse-scale 3D faces with synthesized fine scale geometric wrinkles. I define the wrinkle as the displacement value along the vertex normal direction, and save it as displacement map. The distribution of wrinkles has some spatial characteristics, and deep convolutional neural network (DCNN) is pretty good at learning spacial information across image-format data. I labeled the wrinkle data with its identity and expression vectors. By formulating the wrinkle generation problem as a supervised generation task, I implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. Then I introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. The method can augment an exist 3D face model with more fine-scale details, but to create a realistic human face model is not yet solved. Properly modeling complex lighting effects in reality, including specular lighting, shadows, and occlusions, from a single in-the-wild face image is still considered as a widely open research challenge. To reconstruct an realistic face model from an unconstrained image, I propose a CNN based framework to regress the face model from a single image in the wild. I designed novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting. The outputted face model includes dense 3D shape, head pose, expression, diffuse albedo, specular albedo, and the corresponding lighting conditions.
dc.description.departmentComputer Science, Department of
dc.format.digitalOriginborn digital
dc.identifier.citationPortions of this document appear in: Deng, Qixin, Luming Ma, Aobo Jin, Huikun Bi, Binh Huy Le, and Zhigang Deng. "Plausible 3d face wrinkle generation using variational autoencoders." IEEE Transactions on Visualization and Computer Graphics 28, no. 9 (2021): 3113-3125; and in: Deng, Qixin, Binh H. Le, Aobo Jin, and Zhigang Deng. "End-to-End 3D Face Reconstruction with Expressions and Specular Albedos from Single In-the-wild Images." In Proceedings of the 30th ACM International Conference on Multimedia, pp. 4694-4703. 2022.
dc.rightsThe author of this work is the copyright owner. UH Libraries and the Texas Digital Library have their permission to store and provide access to this work. UH Libraries has secured permission to reproduce any and all previously published materials contained in the work. Further transmission, reproduction, or presentation of this work is prohibited except with permission of the author(s).
dc.subject3D face reconstruction
dc.subjectDeep learning algorithms
dc.subjectSpecular albedo
dc.subjectFacial expressions
dc.subjectFace modeling
dc.subjectWrinkle synthesis
dc.subjectDeep generative models
dc.subjectVariational autoencoders
dc.title3D facial modeling with geometric wrinkles from images
dcterms.accessRightsThe full text of this item is not available at this time because the student has placed this item under an embargo for a period of time. The Libraries are not authorized to provide a copy of this work during the embargo period.
thesis.degree.collegeCollege of Natural Sciences and Mathematics
thesis.degree.departmentComputer Science, Department of
thesis.degree.disciplineComputer Science
thesis.degree.grantorUniversity of Houston
thesis.degree.nameDoctor of Philosophy


License bundle

Now showing 1 - 2 of 2
No Thumbnail Available
4.43 KB
Plain Text
No Thumbnail Available
1.81 KB
Plain Text