Neural machine translation (NMT) systems can improve augmented reality (AR) systems. For example, a smartphone display can translate and text-augment information for travelers, assist with translation of signs, restaurant menus, medical documents, etc., and facilitate real-time translation of foreign-language lectures for students. However, NMT has been neglected in AR research because of its quality and performance requirements. General-purpose translation services, e.g., Google Translate, are used in some AR systems, but they depend on Internet connections and are not well-tuned to providing high-quality translation in specific domains. We propose a laboratory tool prototype that classifies phone camera images, recognizes and translates image text, and augments camera images with translated text in real time. Text from various domains is translated via NMT based on recurrent and convolutional neural networks—with and without subword units. According to bilingual evaluation understudy metrics, the results are very promising, which encourages future applications.
Wołk, Krzysztof, "Incorporating Domain-Specific Neural Machine Translation into Augmented Reality Systems" (2020). PACIS 2020 Proceedings. 14.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.