Multimodal Propaganda Detection via Anti-persuasion Prompt Enhanced Contrastive Learning

Propaganda, commonly used in memes disinformation, can influence the thinking of the audience and increase the reach of communication. Usually logical fallacy, as a kind of popular expression of memes, aims to create a logical reasonable illusion where the conclusion cannot be drawn with the use of correct logic rules. However, this characteristic inherent in memes leads to difficulties for classic multi-label classifiers to understand propagation techniques. To this end, we propose a novel propaganda detection model called Anti-persuasion Prompt Enhanced Contrastive Learning (abbreviated as APCL). First, our APCL reformulates the multi-label classification task by leveraging the category words of propaganda technique based prompt engineering, which is converted into an image-text matching (ITM). Second, prompt engineering is designed with a persuasion prompt template and an anti-persuasion prompt template. The former is to build matched text-image pairs, and the latter is to form mismatched text-image pairs which fit the logical fallacy style of memes. Finally, the propagation technique is predicted based on the distances between the above two prompt templates enhanced texts and an input image. Experimental results on the memes dataset of SemEval2021 task 6 show that our APCL outperforms the state-of-the-art multimodal classification models in terms of F1 measures.