| Title | Microwave Lip Reading of Chinese Mandarin Based on Programmable Metasurface |
| Authors | Jiang, Siyuan Ruan, Hengxin Wang, Zhuo Zhang, Hongrui Zhao, Hanting Li, Lianlin |
| Affiliation | Peking Univ, Dept Elect, Beijing, Peoples R China PengCheng Natl Labrotory, Shenzhen, Peoples R China |
| Issue Date | 2021 |
| Publisher | 2021 IEEE MTT-S INTERNATIONAL MICROWAVE WORKSHOP SERIES ON ADVANCED MATERIALS AND PROCESSES FOR RF AND THZ APPLICATIONS (IMWS-AMP) |
| Abstract | Lip reading is an important way of communication in some specific environment. The traditional lip reading system, which is based on processing and recognizing optical video is becoming not that applicable in areas where the light is obstructed. In order to overcome the shortcomings of optical and video-based methods, microwave lip reading was proposed. The current microwave lip reading system is capable of processing lip reading tasks in the English context. However, this method is not able to distinguish the lip language at the syllable level. In this work, we choose Chinese character as the object of syllable classification task, because each Chinese character consists of only one syllable. We utilize the programmable metasurface to focus the EM wave emitted by the transmitter to the speaker's lip and get the microwave data which contains information of the lip's movement. Then, we use deep learning method to classify the different Chinese character with very high accuracy. At the same time, we prove that the microwave lip reading can detect the movement inside one's mouth cavity. As a result, the microwave lip reading system is able to classify the syllables more precisely than optical method. |
| URI | http://hdl.handle.net/20.500.11897/647073 |
| ISBN | 978-1-6654-2819-4 |
| DOI | 10.1109/IMWS-AMP53428.2021.9643862 |
| Indexed | EI CPCI-S(ISTP) |
| Appears in Collections: | 电子学院 |
