SEMANTIC PREPROCESSOR FOR IMAGE COMPRESSION FOR MACHINES
Mingyi Yang (State Key Laboratory of ISN, Xidian University, Xi’an, China); Luis Herranz (Computer Vision Center); Fei Yang (Universitat Autònoma de Barcelona); Luka Murn (British Broadcasting Corporation); Marc Gorriz Blanch (BBC); Shuai Wan (Northwestern Polytechnical University); FuZheng Yang (Xidian University); Marta Mrak (Queen Mary University of London)
-
Members: FreeSPS
IEEE Members: $11.00
Non-members: $15.00
06 Jun 2023
Visual content is being increasingly transmitted and consumed by machines rather than humans to perform automated content analysis tasks. In this paper, we propose an image preprocessor that optimizes the input image for machine consumption prior to encoding by an off-the-shelf codec designed for human consumption. To achieve a better trade-off between the accuracy of the machine analysis task and bitrate, we propose leveraging pre-extracted semantic information to improve the preprocessor's ability to accurately identify and filter out task-irrelevant information. Furthermore, we propose a two-part loss function to optimize the preprocessor, consisted of a rate-task performance loss and a semantic distillation loss, which helps the reconstructed image obtain more information that contributes to the accuracy of the task. Experiments show that the proposed preprocessor can save up to 48.83% bitrate compared with the method without the preprocessor, and save up to 36.24% bitrate compared to existing preprocessors for machine vision.
