数据集获取（ModelScope）

方式一：MsDataset（Python）

# 前置依赖: pip install modelscope
from modelscope.msdatasets import MsDataset

ds = MsDataset.load("QCRI/COVID-19-disinformation", subset_name="default", split="train")
print(ds)

方式二：CLI（命令行）

pip install modelscope
modelscope download dataset QCRI/COVID-19-disinformation

数据集卡片摘录（源站）

COVID-19 Infodemic Multilingual Dataset

This repository contains a multilingual dataset related to the COVID-19 infodemic, annotated with fine-grained labels. The dataset is curated to address questions of interest to journalists, fact-checkers, social media platforms, policymakers, and the general public. The dataset includes tweets in Arabic, Bulgarian, Dutch, and English, focusing on both binary (misinformation detection) and multiclass classification (different types of infodemic content).

Table of Contents:

• Dataset Overview
  • Languages and Splits
  • File Formats
• Annotations
• Dataset Examples
• Data Statistics
• License
• Citation

Dataset Overview

The dataset consists of tweets related to COVID-19, categorized under two tasks:

1. Binary Classification:
   Detecting whether a tweet contains misinformation.

2. Multiclass Classification:
   Classifying the tweet into specific infodemic categories such as conspiracy theories, harmful content, or false cures.

Languages and Splits

The dataset includes the following languages, each with train, development (dev), and test splits:

• Arabic
• Bulgarian
• Dutch
• English

In addition to individual language datasets, a multilang directory contains a multilingual dataset where tweets from all the above languages are combined in the binary and multiclass formats.

File Formats

The dataset is provided in TSV (Tab-Separated Values) format. Each file contains tweet IDs, labels for seven questions (Q1-Q7), and binary/multiclass annotations. The actual tweet text and associated metadata are not included for privacy reasons.

Directory Structure

• Readme.md: This file
• arabic/, bulgarian/, dutch/, english/: Directories containing language-specific datasets for both binary and multiclass classification.
• multilang/: A directory containing the multilingual version of the dataset.

Each language and the multilingual directory include three sets:

• train
• dev
• test

The *_binary_* files correspond to binary classification, while the *_multiclass_* files correspond to multiclass classification.

Annotations

The dataset contains labels for the following seven questions (Q1-Q7), each related

许可

Apache License 2.0

数据加载示例（表格/文本类）

import pandas as pd, glob, os

files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))

源站：modelscope.cn/datasets/QCRI/COVID-19-disinformation