数据集获取

git clone https://github.com/IBM/Dataset-Epidemiologic-Investigation-COVID19.git

curl -L -o repo.zip https://github.com/IBM/Dataset-Epidemiologic-Investigation-COVID19/archive/refs/heads/master.zip
unzip repo.zip

源站 README 摘录（使用方式）

The Dataset of Epidemiological Case Reports for COVID-19

Introduction

This repository contains a dataset (named ECR-COVID-19) of epidemiological case reports with entity labeling which can be used for information extraction.
The motivation of creating and contributing the dataset is to trigger the research on epidemiologic investigation analysis and automation. COVID-19 is threatening the health of the entire human population. In order to control the spread of the disease, epidemiological investigations should be conducted, to trace the infection source of each confirmed patient and isolate their close contacts. However, the analysis on a mass of case reports in epidemiological investigation is extremely time-consuming and labor-intensive. Using the latest NLP technology to accelerate the information extraction from epidemiological case reports should be a feasible and good way. So we prepared the dataset, meanwhile we also submitted a paper to AMIA 2020, the title is “Accelerating Epidemiological Investigation Analysis by Using NLP and Knowledge Reasoning: A Case Study on COVID-19”.
We collected the epidemiological case reports from Dec 19, 2019 to Feb 7, 2020 from the websites of China CDC and some main-stream news websites. This repository was created from the case reports which were labelled by manual with entities, relations, and events.
Special thanks to China CDC and the subbranchs in local cities. Lots of the data are from their announcements. We also appreciate the following news websites some of the data are from: sina.com.cn, people.com.cn, thepaper.cn and news.163.com etc.
If you use this dataset, please cite our paper:

Wang J, Wang K, Li J, Jiang JM, Wang YF, Mei J, Accelerating Epidemiological Investigation Analysis by Using NLP and
Knowledge Reasoning: A Case Study on COVID-19, AMIA 2020. (submission)

Data Format

Each data file (train.txt, valid.txt, test.txt) in the folder ECR-COVID-19 includes a list of lines. Each line is a string in JSON format and includes a case report and corresponding labels. The following are the description of json keys:
(1) doc_id : the ID of current document
(2) text: the original plain text of the case report
(3) entities: All labelled entities in the “text”. In each entity, there are two numbers and one string, the first number is the starting position of the entity in “text”, the second

数据加载示例（表格/文本类）

import pandas as pd, glob, os

files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))

完整仓库：github.com/IBM/Dataset-Epidemiologic-Investigation-COVID19