Oliver Tueselmann, Kai Brandenbusch, Miao Chen and Gernot A. Fink
Proc. Int. Conf. on Frontiers in Handwriting Recognition, pages 285-299, 2022.
Hyderabad, India
In contrast to traditional keyword spotting, semantic word spotting allows users to search not only for word images with the same transcription as the keyword, but also for concepts which are latent or hidden inside a query. However, it has been shown that mapping word images to semantic representations proves to be a difficult task. As semantic embeddings do not consider syntactic similarity, it is common to find search results with highly ranked semantically similar word images, while words with the same transcription as the search query appear in lower ranks. To counteract this problem, a combination of semantic and syntactic representations usually provides a good trade-off w.r.t. semantic and syntactic metrics. In this work, we present methods for realising a weighted combination of semantic and syntactic information. This allows users to focus more on semantic or syntactic aspects and thus provides new insights to their document collections. Thereby, our proposed methods are not limited to the use of word spotting, but also aim to address the optimization of recognition-free NLP downstream tasks.