Year
Month
(Peer-Reviewed) Joint 3D facial shape reconstruction and texture completion from a single image
Xiaoxing Zeng 曾小星 ¹ ², Zhelun Wu ¹, Xiaojiang Peng 彭小江 ¹, Yu Qiao 乔宇 ¹
¹ Shenzhen Institute of Advanced Technology, ChineseAcademy of Sciences, Shenzhen, China
中国 深圳 中国科学院深圳先进技术研究院
² University of Chinese Academy of Sciences, Beijing, China
中国 北京 中国科学院大学
Abstract

Recent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image.

In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way.

We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.
Joint 3D facial shape reconstruction and texture completion from a single image_1
Joint 3D facial shape reconstruction and texture completion from a single image_2
Joint 3D facial shape reconstruction and texture completion from a single image_3
Joint 3D facial shape reconstruction and texture completion from a single image_4
  • On-chip light control of semiconductor optoelectronic devices using integrated metasurfaces
  • Cheng-Long Zheng, Pei-Nan Ni, Yi-Yang Xie, Patrice Genevet
  • Opto-Electronic Advances
  • 2025-01-07
  • Ferroelectric domain engineering of lithium niobate
  • Jackson J. Chakkoria, Aditya Dubey, Arnan Mitchell, Andreas Boes
  • Opto-Electronic Advances
  • 2025-01-03
  • Smart reconfigurable metadevices made of shape memory alloy metamaterials
  • Shiqiang Zhao, Yuancheng Fan, Ruisheng Yang, Zhehao Ye, Fuli Zhang, Chen Wang, Weijia Luo, Yongzheng Wen, Ji Zhou
  • Opto-Electronic Advances
  • 2025-01-03
  • Direct detection with an optimal transfer function: toward the electrical spectral efficiency of coherent homodyne detection
  • Xingfeng Li, Jingchi Li, Xiong Ni, Hudi Liu, Qunbi Zhuge, Haoshuo Chen, William Shieh, Yikai Su
  • Opto-Electronic Science
  • 2024-12-24
  • Enhanced amplified spontaneous emission via splitted strong coupling mode in large-area plasmonic cone lattices
  • Jiazhi Yuan, Jiang Hu, Yan Zheng, Hao Wei, Jiamin Xiao, Yi Wang, Xuchao Zhao, Ye Xiang, Yong Lei, Wenxin Wang
  • Opto-Electronic Science
  • 2024-12-19
  • High-resolution tumor marker detection based on microwave photonics demodulated dual wavelength fiber laser sensor
  • Jie Hu, Weihao Lin, Liyang Shao, Chenlong Xue, Fang Zhao, Dongrui Xiao, Yang Ran, Yue Meng, Panpan He, Zhiguang Yu, Jinna Chen, Perry Ping Shum
  • Opto-Electronic Advances
  • 2024-12-16
  • High performance laser induced plasma assisted ablation by GHz burst mode femtosecond pulses
  • Jingbo Yin, Yulong Zhao, Minghui Hong
  • Opto-Electronic Advances
  • 2024-12-16
  • Sequential harmonic spin–orbit angular momentum generation in nonlinear optical crystals
  • Yutao Tang, Zixian Hu, Junhong Deng, Kingfai Li, Guixin Li
  • Opto-Electronic Advances
  • 2024-12-16
  • Design, setup, and facilitation of the speckle structured illumination endoscopic system
  • Elizabeth Abraham, Zhaowei Liu
  • Opto-Electronic Science
  • 2024-12-13
  • Ultra-high-Q photonic crystal nanobeam cavity for etchless lithium niobate on insulator (LNOI) platform
  • Zhi Jiang, Cizhe Fang, Xu Ran, Yu Gao, Ruiqing Wang, Jianguo Wang, Danyang Yao, Xuetao Gan, Yan Liu, Yue Hao, Genquan Han
  • Opto-Electronic Advances
  • 2024-10-31
  • Advanced biological imaging techniques based on metasurfaces
  • Yongjae Jo, Hyemi Park, Hyeyoung Yoon, Inki Kim
  • Opto-Electronic Advances
  • 2024-10-31
  • Orthogonal matrix of polarization combinations: concept and application to multichannel holographic recording
  • Shujun Zheng, Jiaren Tan, Hongjie Liu, Xiao Lin, Yusuke Saita, Takanori Nomura, Xiaodi Tan
  • Opto-Electronic Advances
  • 2024-10-23



  • Pencil-beam scanning catheter for intracoronary optical coherence tomography                                Ultrashort pulsed laser induced complex surface structures generated by tailoring the melt hydrodynamics
    About
    |
    Contact
    |
    Copyright © PubCard