Year
Month
(Peer-Reviewed) Joint 3D facial shape reconstruction and texture completion from a single image
Xiaoxing Zeng 曾小星 ¹ ², Zhelun Wu ¹, Xiaojiang Peng 彭小江 ¹, Yu Qiao 乔宇 ¹
¹ Shenzhen Institute of Advanced Technology, ChineseAcademy of Sciences, Shenzhen, China
中国 深圳 中国科学院深圳先进技术研究院
² University of Chinese Academy of Sciences, Beijing, China
中国 北京 中国科学院大学
Abstract

Recent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image.

In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way.

We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.
Joint 3D facial shape reconstruction and texture completion from a single image_1
Joint 3D facial shape reconstruction and texture completion from a single image_2
Joint 3D facial shape reconstruction and texture completion from a single image_3
Joint 3D facial shape reconstruction and texture completion from a single image_4
  • Scaling metasurface design: from reusable unit cells to full-metasurface inverse design
  • Eunbi Jang, Jiwan Hong, Ho-Jun Lee, Seokju Choi, Hyewon Suk, Chanik Kang, Sunae So, Haejun Chung
  • Opto-Electronic Science
  • 2026-10-20
  • Ultrafast all-optical biomimetic encoder for spiking neural networks
  • Qijun Bian, Fangchen Hu, Junlian Jia, Aolong Sun, Sizhe Xing, Boyu Dong, An Yan, Ziwei Li, Jianyang Shi, Nan Chi, Junwen Zhang
  • Opto-Electronic Science
  • 2026-10-20
  • Windmill-shaped multi-unit piezoelectric ceramic tuning fork for cooperatively enhanced photoacoustic methane sensing
  • Botao Liu, Guowei Han, Runqiu Wang, Xuan Yang, Haiyue Sun, Shunda Qiao, Ying He, Angelo Sampaolo, Vincenzo Spagnolo, Yufei Ma
  • Opto-Electronic Science
  • 2026-10-20
  • Ultrafast laser direct writing of highly conductive copper micropatterns
  • Songyan Xue, Chunsan Deng, Mingduo Zhang, Huace Hu, Fayu Chen, Leimin Deng, Jianguo Liu, Hui Gao, Wei Xiong
  • Opto-Electronic Manufacturing
  • 2026-09-30
  • Recent advances in fiber-tip integration based on femtosecond laser 3D printing
  • Yuye Xuan, Dezhi Zhu, Dejun Liu, Ying Wang, Yiping Wang, Changrui Liao
  • Opto-Electronic Manufacturing
  • 2026-09-30
  • Neural holography enabled multi-beam steering for multi-task classifier
  • Lin Wu, Mian Wu, Zhen Li, Ying Qiu, Zichen Liu, Yongquan Zeng, Guozhen Liang, Hanbing Li, Guoxing Zheng, Zhixue He, Xi Xiao, Qi Jie Wang, Jin Tao, Shaohua Yu
  • Opto-Electronic Technology
  • 2026-09-30
  • Laser direct writing of nonlinear metasurfaces on phase-change material Ge₂Sb₂Te₅ for enhanced third harmonic generation
  • Lanpeng Guo, Haochong Liu, Shixin Gao, Ming Tang, Minghui Hong, Tun Cao
  • Opto-Electronic Technology
  • 2026-09-30
  • Nearly vertically emitted nonlinear light sources with high efficiencies enabled by high-Q nearly-degenerate polygon modes
  • Chuntao Li, Yingnuo Qiu, Renhong Gao, Lingling Qiao, Jielei Ni, Min Wang, Jintian Lin, Ya Cheng
  • Opto-Electronic Technology
  • 2026-09-30
  • Multiple degrees-of-freedom optical encryption via pixel-resolved vectorial holography
  • Jianzhong Peng, Haoxian Zhou, Mengzhe Shen, Dapeng Wang, Yu Chen, Renlong Zhang, Fangrui Lin, Xiaoyu Weng, Junle Qu
  • Opto-Electronic Technology
  • 2026-09-30
  • Recent progress on mid-infrared and terahertz dual-comb spectroscopy
  • Zhuoren Wan, Min Li, Binbin Liu, Yu Xia, Yuan Chen, Xiuxiu Zhang, Dongxu Zhu, Yan Dai, Sinan Tao, Hua Li, Ming Yan, Heping Zeng
  • Opto-Electronic Technology
  • 2026-09-30
  • Optical packaging of photonic integrated circuits: bottlenecks and potential solutions
  • Rodrigo Rendeiro, Hugo Neto, Diogo Vaz, Carla Rodrigues, Heming Wei, Jan Nedoma, Antonio Teixeira, Francisco Rodrigues, Carlos Marques
  • Opto-Electronic Technology
  • 2026-09-30



  • Pencil-beam scanning catheter for intracoronary optical coherence tomography                                Ultrashort pulsed laser induced complex surface structures generated by tailoring the melt hydrodynamics
    About
    |
    Contact
    |
    Copyright © PubCard