FakeReveal: Multi-modal_Audio-Visual_feature-based dataset

These two textual representations were then compared to capture inconsistencies between the spoken audio and the lip movements. Three cross-modal textual comparison features were extracted from the Whisper and AV-HuBERT outputs. Before comparison, both transcripts were normalised by expanding contractions, converting text to lowercase, removing punctuation, and standardising whitespace.

The obtained features include Word Error Rate (WER), text similarity score, and correct word count. Together, these features provide complementary measurements of word-level error, overall textual similarity, and exact lexical agreement between audio and visual speech. In a parallel audio-based pipeline, MFCC and spectral features were extracted using the Librosa library.

The resulting 50-dimensional acoustic representation consists of 40 Mel-Frequency Cepstral Coefficients (MFCCs), 1 Spectral Centroid, 1 Spectral Roll-off, 1 Spectral Bandwidth, and 7 Spectral Contrast features. A linear Support Vector Machine (SVM) classifier was then applied to the extracted MFCC and spectral features. Three model-derived outputs were retained for each sample: the prediction label, the prediction probability and the decision score.

The dataset contains two complementary files: FakeReveal_Master_Features.csv, which provides the consolidated feature table and FakeReveal_MFCC_Spectral_Features.pkl, which preserves the 50-dimensional MFCC and spectral feature vectors. The FakeReveal_Master_Features.csv contains the following columns: Video_ID: Identifier. label: class: label, where 0 represents real and 1 represents fake. similarity: textual similarity between the audio transcription and the visual speech transcription.

WER: measuring the word-level discrepancy between the audio and visual speech transcriptions. correct_words_#: Number of words correctly matched between the audio and visual speech transcriptions. mfcc_prediction: Binary prediction produced by the MFCC-based SVM classifier mfcc_probability: Prediction probability produced by the MFCC-based SVM classifier. mfcc_svm_decision_score: decision function score representing the sample's position relative to the classifier's decision boundary.