đź“ś L'article : LIS at CheckThat! 2025: Multi-Stage Open-Source Large Language Models for Fact-Checking Numerical Claims
a été accepté pour publication dans les actes de la conférence CLEF 2025 (Conference and Labs of the Evaluation Forum), un événement international de référence dans le domaine de l’évaluation de différents systèmes d'information.
Ce travail est le fruit des efforts de Quy Thanh LE , doctorant au LIS, lauréat du concours France Excellence Eiffel 2024, sous l'encadrement de :
🧑🏻‍🏫 Ismail Badache (Aix-Marseille Université, INSPÉ, CNRS, LIS, Équipe R2I – Pôle Sciences des Données)
🧑🏻‍🏫 Aznam Yacoub (University of Windsor - School of Computer Science, ON, Canada)
🧑🏻‍🏫 Maamar El Amine Hamri (Aix-Marseille Université, CNRS, LIS, Équipe MoFED – Pôle Analyse et Contrôle des Systèmes)
Cette distinction illustre une collaboration internationale, inter-équipes et inter-pôles réussie au sein du LIS, réunissant les expertises complémentaires des Pôles Sciences des Données et Analyse et Contrôle des Systèmes autour d’une problématique sociétale majeure : la vérification automatique de l’information numérique.
Résumé de l'article :
The fast and large-scale spread of information through social networks and digital platforms has become an important global issue for today's societies, making automated fact-checking necessary. This paper presents the contribution of the R2I and MoFED teams from the LIS Laboratory to the task of automated claim relevance estimation, in the context of the 2025 CheckThat! Lab, specifically Task 3: Fact-Checking Numerical Claims. This task focuses on the verification of claims expressed in Arabic, English, and Spanish, particularly those involving numerical data or temporal references. In this study, we explore the effectiveness of recent open-source large language models (LLMs), such as Mistral and Qwen, for automating the fact-checking of numerical claims. We propose a two-stage pipeline that incorporates these LLMs into the fact-checking process: evidence retrieval and veracity prediction. First, we employ the QwQ-32B model to automatically generate questions from each claim, guiding the retrieval of relevant evidence from the corpus provided for Task 3. Second, we fine-tune the Mistral-Small-24B-Instruct-2501 model using the LoRA (Low-Rank Adaptation) technique to predict the veracity of each claim. This hybrid approach is designed to enhance both the performance and efficiency of the fact-checking pipeline. Despite variations in performance across languages, our method achieved outstanding results, ranking first in all 3 languages: Spanish, English and Arabic. The multilingual nature of the datasets played a crucial role in improving the generalizability of claim validation across linguistic contexts. Our approach obtained macro-F1 scores of 0.503 for Spanish, 0.595 for English and an exceptional 0.960 for Arabic, significantly outperforming the second-best Arabic score of 0.635. These results not only underscore the efficacy of leveraging open-source LLMs for fact-checking, but also contribute to ongoing research in claim detection. They further highlight the importance of language-specific adaptations and the potential of multilingual strategies in the development of robust, automated fact-checking systems.