INFORMATION EXTRACTION FROM PILL BOTTLE IMAGES VIA TEXT STITCHING

Extracting information from a sequence of images of a pill bottle captured through a phone camera can allow customers to record useful entities such as drug name, dosage, instructions etc. This information can enable customers to track their medication adherence and interact with a pharmacy to schedule refills though smart phone applications. However, partial images of pill bottles being text-heavy pose several challenges during image stitching. This is due to the limitations of exist- ing image stitching algorithms, which does not consider text features present in the image. In this work, we propose an end- to-end framework for recognising entities from a sequence of images of pill bottles by using a combination of image and text features. The features are used to generate a text panorama and apply named entity recognition for information extraction. The results obtained on a dataset of pill bottle images show that the accuracy of our proposed framework significantly improves the performance of information extraction from the image sets where state-of-the-art panorama stitching method fails.