数据集获取

git clone https://github.com/ArezoShakeri/MultiConAD.git

curl -L -o repo.zip https://github.com/ArezoShakeri/MultiConAD/archive/refs/heads/main.zip
unzip repo.zip

源站 README 摘录（使用方式）

MultiConAD: A Unified Multilingual Conversational Dataset for Early Alzheimer’s Detection.

This repository provides the complete codebase and data processing pipeline for the following submission:
https://arxiv.org/abs/2502.19208

Summary

Dementia is a progressive cognitive syndrome with Alzheimer’'s disease (AD) as the leading cause. Conversation-based AD detection offers a cost-effective alternative to clinical methods, as language dysfunction is an early biomarker of AD. However, most prior research has framed AD detection as a binary classification problem, limiting the ability to identify Mild Cognitive Impairment (MCI)-a crucial stage for early intervention. Also, studies primarily rely on single-language datasets, mainly in English, restricting cross-language generalizability. To address this gap, we make three key contributions. First, we introduce a novel, multilingual dataset for AD detection by unifying 16 publicly available dementia-related conversational datasets. This corpus spans English, Spanish, Chinese, and Greek and incorporates both audio and text data derived from a variety of cognitive assessment tasks. Second, we perform finer-grained classification, including MCI, and evaluate various classifiers using sparse and dense text representations. Third, we conduct experiments in monolingual and multilingual settings, finding that some languages benefit from multilingual training while others perform better independently. This study highlights the challenges in multilingual AD detection and enables future research on both language-specific approaches and techniques aimed at improving model generalization and robustness.

Repository Structure

It includes scripts for data collection, preprocessing, transcription, translation, and classification experiments using both TF-IDF and E5 Large models across different linguistic settings.

1. Audio Transcription

Scripts related to Automatic Speech Recognition (ASR) and transcript organization:

• ASR_audio_dataset.py – Transcribes audio data.
• ASR_collection.py – Organizes transcripts of audio-based data generated by Whisper ASR into a normalized format.

2. Data Collection & Extraction

Scripts for gathering and structuring data from 16 different datasets into a standardized format:

• collection.py – Defines the **normalized c

数据加载示例（表格/文本类）

import pandas as pd, glob, os

files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))

完整仓库：github.com/ArezoShakeri/MultiConAD