数据集获取

git clone https://github.com/cyang03/CHECKED.git

curl -L -o repo.zip https://github.com/cyang03/CHECKED/archive/refs/heads/master.zip
unzip repo.zip

源站 README 摘录（使用方式）

CHECKED

The first Chinese COVID-19 fake news dataset based on the Weibo platform. Please check out our paper here.

Notice

We care about users’’ privacy and made (will keep making) efforts to protecting it.

• For microblogs: We released the hashed id instead of the original id of microblogs.
• For users: We did not make the user_name public, which enables to identify Weibo users. In addition, we released the hashed user_id instead of the original user_id.
• Please use the CHECKED data only for academic research.
  Update:
  We corrected a small amount of dates displayed in the comment sections of two microblogs, which occured due to unknown errors during the automatic information extraction process. We accomplished this by checking the dataset manually. We appreciate the user for pointing this issue out. If similar problems are to be found by future users, please contact us as we will fix them immediately. (August 22, 2021)
  We add analysis as a new keyword for each microblog labeled as fake. analysis contains the expert analysis and justification, which details the news falseness. Please check out the dataset folder for details. We also provide benchmark results of FastText, TextCNN, TextRNN, Att-TextRNN, and Transformer using the CHECKED data in predicting fake news. Please check out the baseline folder for details. (June 9, 2021)

Overiew

This repository includes three folders, named as (1) dataset, (2) code, (3) baseline, respectively.

• dataset: This folder contains the CHECKED data in both json and csv format and the list of keywords used to determine whether a microblog is relevant to COVID-19 or not. Specifically,
  • fake_news: This folder includes 344 fake microblogs (in json format).
  • real_news: This folder includes 1760 real microblogs (in json format).
  • *.csv: These csv files are converted from json files in the fake_news and real_news folder.
  • keyword_list.txt: This file includes all the keywords that we use t

数据加载示例（表格/文本类）

import pandas as pd, glob, os

files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))

完整仓库：github.com/cyang03/CHECKED