Year
Month
(Preprint) From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network
Yuxin Wang 王裕鑫 ¹, Hongtao Xie 谢洪涛 ¹, Shancheng Fang ¹, Jing Wang ², Shenggao Zhu ², Yongdong Zhang 张勇东 ¹
¹ University of Science and Technology of China
中国科技大学
² Huawei Cloud & AI
华为云人工智能
arXiv, 2021-08-22
Abstract

In this paper, we abandon the dominant complex language model and rethink the linguistic learning process in the scene text recognition. Different from previous methods considering the visual and linguistic information in two separate structures, we propose a Visual Language Modeling Network (VisionLAN), which views the visual and linguistic information as a union by directly enduing the vision model with language capability. Specially, we introduce the text recognition of character-wise occluded feature maps in the training stage. Such operation guides the vision model to use not only the visual texture of characters, but also the linguistic information in visual context for recognition when the visual cues are confused (e.g. occlusion, noise, etc.).

As the linguistic information is acquired along with visual features without the need of extra language model, VisionLAN significantly improves the speed by 39% and adaptively considers the linguistic information to enhance the visual features for accurate recognition. Furthermore, an Occlusion Scene Text (OST) dataset is proposed to evaluate the performance on the case of missing character-wise visual cues. The state of-the-art results on several benchmarks prove our effectiveness.
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_1
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_2
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_3
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_4
  • On-chip light control of semiconductor optoelectronic devices using integrated metasurfaces
  • Cheng-Long Zheng, Pei-Nan Ni, Yi-Yang Xie, Patrice Genevet
  • Opto-Electronic Advances
  • 2025-01-07
  • Ferroelectric domain engineering of lithium niobate
  • Jackson J. Chakkoria, Aditya Dubey, Arnan Mitchell, Andreas Boes
  • Opto-Electronic Advances
  • 2025-01-03
  • Smart reconfigurable metadevices made of shape memory alloy metamaterials
  • Shiqiang Zhao, Yuancheng Fan, Ruisheng Yang, Zhehao Ye, Fuli Zhang, Chen Wang, Weijia Luo, Yongzheng Wen, Ji Zhou
  • Opto-Electronic Advances
  • 2025-01-03
  • Direct detection with an optimal transfer function: toward the electrical spectral efficiency of coherent homodyne detection
  • Xingfeng Li, Jingchi Li, Xiong Ni, Hudi Liu, Qunbi Zhuge, Haoshuo Chen, William Shieh, Yikai Su
  • Opto-Electronic Science
  • 2024-12-24
  • Enhanced amplified spontaneous emission via splitted strong coupling mode in large-area plasmonic cone lattices
  • Jiazhi Yuan, Jiang Hu, Yan Zheng, Hao Wei, Jiamin Xiao, Yi Wang, Xuchao Zhao, Ye Xiang, Yong Lei, Wenxin Wang
  • Opto-Electronic Science
  • 2024-12-19
  • High-resolution tumor marker detection based on microwave photonics demodulated dual wavelength fiber laser sensor
  • Jie Hu, Weihao Lin, Liyang Shao, Chenlong Xue, Fang Zhao, Dongrui Xiao, Yang Ran, Yue Meng, Panpan He, Zhiguang Yu, Jinna Chen, Perry Ping Shum
  • Opto-Electronic Advances
  • 2024-12-16
  • High performance laser induced plasma assisted ablation by GHz burst mode femtosecond pulses
  • Jingbo Yin, Yulong Zhao, Minghui Hong
  • Opto-Electronic Advances
  • 2024-12-16
  • Sequential harmonic spin–orbit angular momentum generation in nonlinear optical crystals
  • Yutao Tang, Zixian Hu, Junhong Deng, Kingfai Li, Guixin Li
  • Opto-Electronic Advances
  • 2024-12-16
  • Design, setup, and facilitation of the speckle structured illumination endoscopic system
  • Elizabeth Abraham, Zhaowei Liu
  • Opto-Electronic Science
  • 2024-12-13
  • Ultra-high-Q photonic crystal nanobeam cavity for etchless lithium niobate on insulator (LNOI) platform
  • Zhi Jiang, Cizhe Fang, Xu Ran, Yu Gao, Ruiqing Wang, Jianguo Wang, Danyang Yao, Xuetao Gan, Yan Liu, Yue Hao, Genquan Han
  • Opto-Electronic Advances
  • 2024-10-31
  • Advanced biological imaging techniques based on metasurfaces
  • Yongjae Jo, Hyemi Park, Hyeyoung Yoon, Inki Kim
  • Opto-Electronic Advances
  • 2024-10-31
  • Orthogonal matrix of polarization combinations: concept and application to multichannel holographic recording
  • Shujun Zheng, Jiaren Tan, Hongjie Liu, Xiao Lin, Yusuke Saita, Takanori Nomura, Xiaodi Tan
  • Opto-Electronic Advances
  • 2024-10-23



  • NetGraph: An Intelligent Operated Digital Twin Platform for Data Center Networks                                IGNNITION: fast prototyping of graph neural networks for communication networks
    About
    |
    Contact
    |
    Copyright © PubCard