MTLFND: Multimodal fake news detection using attention mechanism and transfer learning

Abstract

In the era of increasing information, fighting fake data has become a topmost significance. The Multimodal Transfer Learning framework for False News Detection (MTLFND) is suggested in the paper, presenting an innovative approach to enhance accuracy and robustness in recognizing misleading information across diverse media formats. While existing approaches rely on performing multimodal fusion techniques, the proposed method addresses this by introducing a novel MTLFND module. The cross-modal similarity between textual and graphical features is captured by this module, intelligently leveraging pre-trained knowledge. The extracted features are dynamically weighed and combined with this similarity information, with relevant details from both modalities being prioritized during classification. This is the first state-of-the-art method to employ redundancy reduction and modality-wise attention to further refine the multimodal features before feeding them into the final classifier according to the best of our knowledge. Numerous experiments show that the suggested model is successful, surpassing other leading methods in accuracy across various datasets. The value of multimodal transfer learning models for enhanced flexibility in feature selection is highlighted by this advancement, paving a way for further research in the promising path.