数据集获取

git clone https://github.com/sag111/COVID-19-tweets-Russia.git

curl -L -o repo.zip https://github.com/sag111/COVID-19-tweets-Russia/archive/refs/heads/master.zip
unzip repo.zip

源站 README 摘录（使用方式）

COVID-19-tweets-Russia

This repository provides code and additional materials of the paper: “The Russian language corpus and a neural network to analyse Internet tweet reports about Covid-19”.
One of the features we analyze is the number of tweets where Internet users report having Covid-19 symptoms. Extraction of such tweets from the Internet is complicated due to the lack of a Russian-language tweet dataset which is needed for training the machine learning models to automatically extract this category of tweets, so we focused on creating the first tool for the classification of the coronavirus mentioning in the texts of Russian-language tweets.
Data

We collected three types of data:

1. Annotated part (9,860: train 5,916, valid 1,972, test 1,972);
   We collected Russian-language tweets containing the word “covid” that were published from 01 Mar 2020 to 01 Mar 2021 and randomly selected 10,000 messages from them for a manual annotation.
   Annotators labeled messages, dividing them into 5 classes:
   • potential_covid_high_confidence potentially has coronavirus with high probability;
     -potential_covid_low_confidence potentially has coronavirus with low probability;
   • had_covid_high_confidence had coronavirus with high probability;
   • had_covid_low_confidence had coronavirus with low probability;
   • other mention coronavirus but don’'t refer to a potential case of the author or his\textbackslash her relatives
2. Unlabeled part with tweets from users from large regions of Russia (total 705,453 tweets for a 15 largest regional center of Russia).
   The search was provided using different word forms of 58 manually selected keywords on Russian related to the topic of coronavirus infection (including: “PCR”, “pandemic”, “self-isolation”, etc.).
   The affiliation of a tweet to one of the regions was determined in two ways:
   • the text of the message directly mentions the capital of the region;
   • there is a capital of the region in the field “home region” of the author’'s account.
3. Unlabeled part with tweets about covid for Language Model training (total 1,979,247 tweets);
   The data on keyword “covid” was expanded with texts containing other words often occurred in hashtags on the Covid-19 pandemic: “covid”, “stayhome”, and “coronavirus” (here, these are translations of Russian words into English). This part includes all unique Russian-lan

数据加载示例（表格/文本类）

import pandas as pd, glob, os

files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))

完整仓库：github.com/sag111/COVID-19-tweets-Russia